That is an excellent question!

The answer is simple: The same way you do it in the non Grid Computing world!

The way I do TDD is to push away external systems, libraries, components etc, by wrapping them with Interfaces. This gives me the abilitity to Test/Design the behaviour of an application without external dependencies having too much influence on the overall design.
With Unit level tests I can then substitute the Interface to the external system (Grid in this case) with a Mock or Stub implementation. This provides a number of advantages:

• I can run all of the unit tests without having to deploy/run a grid etc.
• The tests run very quickly, fail fast, as they do not depend on potentially slow external systems such as databases etc.
• Most of the code can run anywhere, it does not have to run on a Grid.
• ...

Of course this approach doesn't cover case where you really want to know that it works when everything is plumbed together. That is what Functional tests are for.

Functional tests should test from top to bottom with real dependencies plumbed in. Further, they should run as a seperate stage in your build pipeline so they don't slow down the feedback loop. Slow feedback == frustrated developers == lower moral == short cuts == a mess!

So the moral of the story: TDD on a grid application is no different to TDD in any application. Fail fast and be happy!