Run Google Dataproc 3.0, Apache Spark 4.1, Hadoop 3.5, and Hive 4.2 standalone in Docker. A complete guide for developers and AI agents with single-container and 3-node cluster modes.

1 points•jaysen_apache•about 3 hours ago•0 comments•
Google does not provide official docker image for developers or AI agents to run dataproc cluster locally. so everybody need to spin up for even dev/CI testing,etc..

now you or your AI agent can: docker run --rm agentcloud/dataproc:3.0.0-debian13 self-test

saves time and cost.

it basically packages the same component version and make the hadoop and spark to developer.

Here are docker images for other dataproc versions:

https://hub.docker.com/r/agentcloud/dataproc/tags

here is full blog for more use-cases: https://local.cloud/blog/run-dataproc-locally-docker/

0 comments

No comments yet.

Related stories