Black is identical on PS2 and Xbox.This is probably just mugging for the cameras, so to speak, but I've heard nothing but good things about the visuals in Black. If it's true -- that they were competing with the Xbox 360 -- then perhaps thePlayStation 2 really is difficult to program, but flexible and powerful enough to do amazing things.
At Criterion, we pride ourselves on our technology, particularly for PlayStation 2. While the Xbox version sports pretty much the same state-of-the-art audio, visuals, physics etc, the real challenge was in coaxing this performance out of PS2.
We're astonished how far we managed to push the system with Black. We realized early in development that our main competition in terms of a visual benchmark would be on Xbox 360, and we worked hard to beat the first titles you see on that system. Short of higher resolution, Black's the best-looking FPS we've seen.
Will programmers ever push the GameCube and the original Xbox that hard? Maybe Twilight Princess on the Cube will come close, but I'm not sure that the Xbox was ever truly pushed as hard as the PlayStation 2 has been.
It'd be interesting to know why certain systems are successful and pushed to their limits and others aren't. Is it just market forces? The continuing interest in Atari 2600 and Commodore 64 hacking indicates otherwise, as interest in those systems has far outlived the original companies. It's not just the challenge of a difficult system either, or the Saturn would have lived past Panzer Dragoon Saga and Burning Rangers. Nor is it the ease, or the Dreamcast would have lived longer. Then there are systems like the PSOne which were apparently easy to program for, but were also pushed very, very hard even within their commercial lifetimes.
It's largely due to market forces, as you state. The PS2, with the largest install base, gets (or, rather, got) first dibs from publishers. With more developers being told to work on the PS2, there's more incentive to 'compete' and add that special technical flash to the game that seperates it from the rest of the pack.
It doesn't seem like the XBox will ever get developers pushing the HW at the same level, which is unfortunate, since it's basically a fixed architecture PC (complete with MS developer tools), which seems like an ideal platform for experimentation. But at this point, MS is pushing everyone to develop for the 360, and forget about the XBox.
Resident Evil 4 probably pushed the Gamecube a bit, but only because Shinji Mikami was in love with the machine. Capcom probably put much more effort into pushing the PS2 to catch up. (The PS2 version doesn't look that bad unless you do side by side comparisons.)
The PS2 is going to be pushed further than the GC or XBox just because it has the largest user base and perceived best market. (This is debatable, as there are multiplatform game figures that have circulated that have shown the GC to have comparable sales on at least some titles, and rarely surpassing PS2 sales. But with no sales context on the figures.)
That the PS2 is considered the weakest system may only encourage it to be pushed harder. Companies want the PS2 version to look great, because they see it as their best market. They don't care so much if the Xbox and GC audiences complain their versions are held back to match the PS2. But they certainly don't want to anger the PS2 market by making it look like the PS2 version is too inferior.