MaxCompute Studio

Compatible with: IntelliJ IDEA PhpStorm WebStorm PyCharm RubyMine AppCode CLion GoLand DataGrip Rider
Mar 15, 2019
412 028

MaxCompute is a big data processing platform developed by Alibaba Cloud independently. It is a fast and cloud-based big data solution that supports multiple distributed data storage and processing models, which can provide massive data warehouse and big data modeling service.

MaxCompute Studio is a plugin for IntelliJ platform allowing data developers works with MaxCompute platform including authoring SQL scripts, UDF extensions, MapReduce programs and other functions like local debugging, data browsing and uploading/downloading, job browsing and analytics, etc.

Features include:

  • MaxCompute SQL language support
  • MaxCompute function development
  • MaxCompute data management
  • MaxCompute job management

Recent change notes

Changes in Version 3.1.1:

  • New Feature:
    • support job operator level metric diff.
    • support specify macro parameter at sql editor toolbar.
    • show sub status and duration when click status link in job details.
  • Enhancement:
    • job analysis refactor.
    • job DAG graph robustly.
    • table partition info column readable and sortable.
  • Bug Fix:
    • table partition not exist error in sql editor.
    • NPE when click a not exist table in sql editor.
    • session job can't display query script.

Changes in Version 3.1.0:

  • New Feature:
    • support job diff in job explorer, sql history, open logview directly.
    • udf local run support constant.
  • Enhancement:
    • use text editor to show fuxi instance stdout/stderr/debug info.
    • support open entity button at project explorer toolbar.
    • show column not null in pot graph.
    • some icon and text changes in job details.
    • adjust job graph operation‘s metric display order.
  • Bug Fix:
    • NPE when init JobRefreshAction.
    • import data to partition table failed.
    • comment should not treat as macro when job submit.
    • proxy not use cache to get user info.
    • udf remote debug break.
    • incompatibility error in IntelliJ idea 2017.2.*.
    • job analysis tab can't display long tail/skew instance name.
    • job detail tab fuxi instance status filter‘s dimension.
    • job graph edge schema may be not match.
    • fuxi task time may be inconformity.

Changes in Version 3.0.5:

  • Enhancement:
    • adjust metric info order in operation graph.
    • user can config resource file path when deploy to server.
  • Bug Fix:
    • test tunnel endpoint connection failed.
    • check UDF resolve argument failed before idea 2017.
    • open logview may failed.
    • cut off compile error message if too long.

Changes in Version 3.0.4:

  • New Feature:
    • check UDF annotate arguments(both Java and Python).
    • show executor host and hbo plan in job details.
  • Enhancement:
    • use csv data format to export grid data.
    • use pot to draw job graph if possible when multiple jobs.
    • highlight and code completion for Parameterized View.
    • show fuxi instances at the same time when click 'open in fuxi task tab' in job graph.
  • Bug Fix:
    • import d2 nodes number format exception.
    • copy meta failed if table lifecycle is -1.
    • MaxCompute not support block comment.
    • re-init if user run script on IntelliJ toolbar.
    • NPE when show resource privilege.
    • open table in meta failed.

Changes in Version 3.0.3:

  • New Feature:
    • show plan tab at job details page.
    • show record num at progress indicator when table import/export.
  • Enhancement:
    • sql code completion and format enhancement for UDT.
    • sql format enhancement for case when and so on.
    • show table data header initially and add import/export button in table details page.
    • use plain text when not sql task in job details page.
  • Bug Fix:
    • List d2 folder failed.
    • job graph bug fix like replay not shown and so on.
    • type system selection not work.
    • preview sql format break on setting page.
    • modify project tunnel endpoint not work.
    • console not support multi line in windows.
    • ArrayIndexOutOfBoundsException in job list.
    • apache CollectionUtils NoClassDefFoundError.
    • open logview url panel layout.

Changes in Version 3.0.2:

  • New Feature:
    • job list: range submit time select, quota group id hyperlink, performance enhancement.
    • show cluster columns info when table code completion.
  • Enhancement:
    • instance basic info ui update and performance enhancement.
    • not show partition tab when showing not partitioned table details.
    • multi keyword suggestion enhancement.
    • always use sql task when script mode.
    • use uniform text alignment in sql result table.
    • treat waitingPosition == 0 as running.
    • not show priory and cpu for public cloud project job list.
  • Bug Fix:
    • use new ak to init MaxCompute connection when user update ak.
    • prepare set flag for console command in sql editor.
    • job detail result tab table name duplicate and said running when when terminated.
    • job detail not show summary and json summary tab when no data.
    • generate ddl illegal character on windows.
    • view multi partition table exception.
    • duplicate text attribute name error.
    • code completion get table desc NPE.
    • table schema NPE in udf run configuration.

Changes in Version 3.0.1:

  • New Feature:
    • support batch set MaxCompute project for script folder.
    • support grant sql generate wizard.
  • Enhancement:
    • log exception stack trace when compile error.
    • remember user input on udf deploy.
  • Bug Fix:
    • edge schema display bug in operation graph.
    • can't display char type schema bug.
    • can't display pot graph when there is buffered operation.
    • compile error in studio when multi partitions.
    • string in java file compile error.
    • date format inspect error.
    • create function NPE when none MaxCompute project.
    • MR main class check logic.

Changes in Version 3.0.0:

  • New Feature:
    • support write authorization statement, show user privileges and diagnosis auth exception message.
    • use web view to draw sql pot graph and performance enhancement.
    • support copy project meta.
    • support show project detail.
    • support variable in sql editor.
    • support input/output byte hot graph and display edge schema in operation level graph.
    • Show optimizer relation plan in analysis page for sql instance.
    • Show formatted pot/midlot in analysis page and support text search.
  • Enhancement:
    • multi keyword code completion enhancement.
    • set and setproject code completion.
    • with menu item icon in project explorer.
    • editor support merge small files statements.
    • give user choice to format json when view cell detail.
    • sql select result searchable.
    • convert 'create table as' to 'insert overwrite table' intention action.
    • support specify your own python path to resolve python UDF.
    • use typeinfo to display edge schema instead of type which is more accuracy.
    • support auto fill in folder path when create new sql to sync to D2(Internal version only).
    • support display multiply operation metric message in operation graph.
  • Bug Fix:
    • fix running job graph can't auto update bug.
    • Instance detail table support scroll selected row to visible view after re-sort.
    • editor hang when external table without resources.
    • jar is not supported to sync dataworks.
    • function is ambiguous with ().
    • udf run configuration column trim blank.
    • use gutter button to execute explain statement bug.

Changes in Version 2.9.5:

  • New Feature:
    • Provide new designed table builders.
  • Enhancement:
    • Support user specify a job name when submit script.
    • Sql format enhancement: with statement, loop statement.
    • Auto detect base project when sync new sql to d2 (Alibaba internal only).
  • Bug Fix:
    • Drop view failed in studio project explorer.
    • Editor hang when writing external table sql without using resources.
    • Function return type in sql editor code completion.
    • Fix sync to D2 bug(Alibaba internal only).

Changes in Version 2.9.4:

  • New Feature:
    • Add resource and create function can sync to D2(Alibaba internal only).
    • Detect jdk version and suggest user update jdk.
  • Enhancement:
    • User can see MaxCompute job list in sql running log.
    • Add keystroke(shift ctrl F5) for meta sync.
  • Bug Fix:
    • Interactive query should not available for public cloud project now.
    • Fix login incompatible on different intellij version.
    • Fix bind odpscmd jdk version incompatible on different intellij version.
    • Fix repeatedly open console in studio.
    • Fix job graph bug.

Changes in Version 2.9.3.1:

  • Bug Fix:
    • Show project and endpoint in MaxCompute project combobox.
    • Can't find resource bundle error in English environment.

Changes in Version 2.9.3:

  • New Features:
    • Support add project connection with same name(different endpoint).
    • Support scratch script file.
  • Enhancement:
    • Disable select result export.
    • Disable export result data in job detail.
    • Table select combobox editable in UDF run configuration.
    • Enhance sync meta performance and show user synced number.
    • Enhance job graph ui, change progress graph to four part to display percentage.
    • Support change edge type on job graph and fix node size when rerun job.
    • Update odpscmd to 2.9.1 and bind odpscmd start jdk with intellij boot jdk.
  • Bug Fix:
    • Create view statement mistake for console command.
    • Some bug fixes.

Changes in Version 2.9.2:

  • New Features:
    • Support execute console command in sql editor.
  • Enhancement:
    • Show task detail info in job graph.
    • Show schema info between tasks in job graph.
    • Format date column in sql query result table.
    • UDT 'new' keyword suggestion and highlighting.
  • Bug Fix:
    • Error highlighting broken in sql editor.

Changes in Version 2.9.1:

  • New Features:
    • UDT editor support.
  • Enhancement:
    • Scroll selected row to visible viewport after job detail table being sorted.
    • Query result table support sort by column type.
    • Display fuxi job layer first when open multiply fuxi job instance.
    • Support execute one statement with set.
    • Support sync big project.
    • Add summary tab in job detail.
    • Add I/O info between operators in job graph.
  • Bug Fix:
    • Get user profile may fail.
    • UI hang after user login.
    • Ddl job graph disappear after refresh.
    • NPE when new script has macro parameters.
    • Python UDF test tab character trim.
    • Exception when modify python udf main class name.

Changes in Version 2.9.0:

  • New Features:
    • Support interactive query in studio
    • Support Alipay projects (Using proxy for Alibaba internal version only)
    • Support User login and list related projects (Using SSO for Alibaba internal version only)
    • Support to inspect SQL and quick fix in editor
    • Support graph computing plugin development
  • Enhancement:
    • Remote debug performance enhancement(Alibaba internal version only)
    • Optimize job detail table view layout and latency display format
    • User can config column separator when import/export table
    • Data Preview and SQL Query Result table can select column; format the date column of data preview grid
    • Show function overload signatures in editor for code completion
    • Show progress bar for table data preview task and allow user to cancel it
    • Enhance integration with Dataworks (Alibaba internal version)
    • Remember recently opened logview file
  • Bug Fix:
    • Fix bugs in job graph (eagle eye disappear sometimes, remember select type on change graph level)
    • Remove error notification when opening table view in project explorer.
    • Fix editor highlight bug on Table Name in Function Parameter is not Highlighted
    • Some other fixes