A bundle of plugins for data engineers and other specialists engaged with big data workloads. Installed in your favorite JetBrains IDE, Big Data Tools helps develop, visualize, debug, and monitor big data pipelines built in Scala, Python, and SQL.
Use Big Data Tools for:
- Exploratory analysis, visualization, and prototyping jobs in Zeppelin notebooks.
- Running and monitoring Spark or Flink jobs directly from your IDE.
- Working with Amazon EMR clusters.
- Viewing big data files, such as CSV, Parquet, ORC, and Avro.
- Producing and consuming messages with Kafka.
- Previewing Hive Metastore databases.
- Getting insights about your Hadoop environment.
Built-in tools and integrations:
- Supported languages: Scala, Python, SQL.
- Notebooks: Zeppelin.
- Monitoring: Hadoop, Kafka, Spark, Hive Metastore, Flink, AWS Glue.
- Remote file storages: AWS S3, Google Cloud Storage, Microsoft Azure, Tencent Cloud Object Storage (COS), DigitalOcean Spaces, Alibaba OSS, Hadoop Distributed File System (HDFS), and more.
- File systems: HDFS, Local, SFTP.
- Data processing platforms: AWS EMR.
Bartek Rychlicki
26.01.2025Superb, makes my life so much easier in S3.