Run Google Dataproc 3.0, Apache Spark 4.1, Hadoop 3.5, and Hive 4.2 standalone in Docker. A complete guide for developers and AI agents with single-container and 3-node cluster modes.
Google does not provide official docker image for developers or AI agents to run dataproc cluster locally. so everybody need to spin up for even dev/CI testing,etc..
now you or your AI agent can: docker run --rm agentcloud/dataproc:3.0.0-debian13 self-test
saves time and cost.
it basically packages the same component version and make the hadoop and spark to developer.
Here are docker images for other dataproc versions:
https://hub.docker.com/r/agentcloud/dataproc/tags
here is full blog for more use-cases: https://local.cloud/blog/run-dataproc-locally-docker/
0 comments
No comments yet.
Related stories
- Hacker News · 1 points · 4 days ago
- Hacker News · 19 points · about 15 hours ago
- Hacker News · 421 points · 8 days ago
- Hacker News · 3 points · 4 days ago
- Hacker News · 3 points · 9 days ago
- Hacker News · 1 points · 7 days ago