Originally meant to be a short LinkedIn post but exceeded the word limit! Written unassisted by yours truly 🙂
A few weeks back I was kindly invited to give Plotly Studio a test run. Plotly Studio is an AI-Powered BI tool in early access and you can try it for yourself here : https://plotly.com/blog/introducing-plotly-studio/
Positives
- Designed by Data Analysts for Data Analysts and it shows
- Potentially useful for templating, prototyping and ideation for experienced Plotly and Dash Users
- Potentially useful for quick explorations of simple “boilerplate” datasets
- Don’t have to interact with a chatbot
Points of Friction
- Persons not experienced in Dash, Python and Plotly will experience difficulties in its use. A user should have a decent understanding of the limits and functionality of Dash, Python and Plotly to provide good instructions, to debug, tweak and incrementally build. A user should have a decent understanding of Dash, Python and Plotly to assess the validity of the output. For more nuanced take see Kurt Buhler’s article here (my thoughts are in alignment) : https://shorturl.at/eNTow
- Data analysis is akin to building knowledge – knowledge is built incrementally or cumulatively. Whilst the designers of the product say they prefer to work backwards – I prefer to take component parts and work forwards toward a conclusion – this method causes less confusion, you don’t have to guess assumptions, because you put the assumptions in. Thinking with component parts is also why a grammar of graphics approach to data visualisation produces better results. See also : Brandolini’s law – https://tinyurl.com/469ssb3z
- In terms of design and build of an application, as opposed to data analysis, I will add to the analogy the product designers expressed of finding it easier to work from a draft – this might be the case, however, some people may find it easier to design from a blank canvas and develop from a draft they produced. They may or may not find it easier to develop from a draft someone else produced. Drafting is different from templating – working from a template can be easy, when the template approximates the desired end-state.
- Communicating about visual communication without visual communication is difficult (a picture says a 1000 words, but try describing a picture in 1000 words). It is also rather backwards… See also : You Don’t Need Words to Think https://tinyurl.com/mrc8x49t
- Data visualisation is a creative process. The generated dashboards are confined to certain styles and best practices, e.g. filters at top, callout values, 4 main visuals and a table at the bottom. Great template guidance for entry-level analysts, but very much confines and restricts creative thinking and the opportunity for novelty. Novelty being an enabler for more effective data visualisation solutions. See also : Creative Data Visualisation with Power BI https://shorturl.at/XCCRF
- Writing prompts or learning to prompt effectively with non-deterministic results is sometimes as much or more effort as writing the code yourself
- Plus more that would necessitate a longer blog article.
Still has all the problems that AI-Powered Apps have
- Trust
- Black box with little understanding of what it has been trained on
- Confabulations
- Rework
Other considerations
- What happens if underlying AI model regresses or collapses?
- What if costs of use of underlying AI model increases?
- Security
- Reliability
- Future Roadmap and Future Supportability
Personal opinion
Impressed by the design thinking going into the Plotly Studio toolset. However, would like to see an intermediary between prompt and code, like Plotly Chart Studio or Graphic Walker. See also https://kerrykolosko.com/my-biggest-wish-for-the-power-bi-suite/.
Opportunities would be to have more templated options for the dashboard layout, and ability to view and edit the initial prompts (e.g. the requirements specification). Perhaps even a blank canvas whereby charts could be added incrementally to build a story, rather than all at once.
I personally couldn’t get by on prompts alone. I much prefer to expend my time and effort to learn these languages to an expert level or sit and wish for a low-code tool like Power BI or Graphic Walker to extend it’s data visualisation and statistical analysis functionality.
A longer reaction video here :
(note video not designed to be engaging, but to give a truer sense of the effort and friction points in using this tool in everyday workflows — rather like watching someone use an iPhone for the first time — there are less “ums” and “ahs” further along the video 🙂 )