Hey Fred, there's the new Hadoop Agent available for running Pig, Hive, and Map Reduce jobs. Here's a
link to the documentation for it.
There's also a plugin on the
Marketplace with template jobs for running
Hadoop from the Command Line.
Unfortunately I haven't done anything with Spark yet.