Espalier

Research project by Matt McCutchen, Shachar Itzhaky, Daniel Jackson, Willow Jarvis, Yang Yan, and Sophie Mori (espalier@mit.edu) of the Software Design Group, with help from various others.

Espalier (formerly Object Spreadsheets) is a new computational paradigm that combines the usability advantages of spreadsheets with SQL-like expressive power, providing a way to build a wide class of interactive applications more easily than with existing tools.

Shortcuts: paper, demo server, source repository, all materials, UROP/MEng projects

Introduction

Nearly every organization is responsible for processes that involve collection and manipulation of structured data subject to certain rules; in CSAIL and EECS alone, examples range from a milk pool that tracks whose turn it is to buy milk next to the graduate admissions process. The Espalier tool can be used to build organizational applications that support these processes by letting users view and update data subject to the applicable constraints. Espalier seeks to provide an integrated visual environment to construct the schema, data, and logic that is natural enough to appeal to end-user developers yet general enough to conveniently express arbitrary logic. It is the first spreadsheet-based tool to support editing of hierarchical structured data (important in many applications) and a formula language with SQL-like expressive power.

There is a long history of tools for building organizational applications (HyperCard, FileMaker, App2You, QuickBase, Zoho, Airtable, ...), but to date they have seen limited adoption; we suspect this is because none has achieved a compelling enough combination of naturalness and generality. Instead, many organizations continue to use general-purpose application frameworks such as Rails and Meteor (which demand a depth of technical understanding disproportionate to the problem) or to cobble together tools such as email, spreadsheets and online forms that are easy to use but cannot fully express the application logic (leaving manual work and crude security controls). With Espalier, we hope to offer developers the best of both worlds for low- to medium-complexity organizational applications.

Status

The project has been mostly on hold since I left SDG in November 2018. As of August 2021, I currently have other priorities, but I would be interested in working more on Espalier at some point if a promising enough line of work arises; please do email me if you might be interested in collaborating. SDG is graciously continuing to host the public demo server for now; please let me know if it is not working and I'll try to fix it. ~ Matt

As of July 2018, our prototype tool supports all basic spreadsheet functionality, plus updatable views and application-specific action macros, which support our approach to developing application logic. It includes both (1) the "new" spreadsheet UI, which lets developers customize the data layout in two dimensions, and (2) the "old" spreadsheet UI, which uses the standard nested table layout. The new UI is rough at this point and is the focus of current work, while the old UI is relatively mature and is in maintenance mode. Our newest approach to building applications is available on the "public-demo-server" branch of the source repository (the required functionality is not yet merged to "master") and uses styled spreadsheets built in the new UI for both the master data set and the application UI. We have one example application demonstrating this approach: a grocery store that lets users browse items and add them to a cart. Since we have only this one example (so far) of the new approach, we are continuing to maintain an older set of example applications built using the old UI and the Mavo templating engine for the application UI. Many features to improve the usability of the tool have yet to be built.

We're in the process of renaming the project from Object Spreadsheets to Espalier, so you'll still see the Object Spreadsheets name in some places.  The name Espalier refers to a method of growing trees in specific shapes, which is a metaphor for how our new UI displays a hierarchical data set in a user-defined layout.

We would welcome your feedback on this work, and we'll provide support for the prototype subject to our research priorities; in either case, please email us.

Available materials

Addendum to the instructions to run your own Espalier server

As of this writing (2022-03-19), the instructions in the source repository unfortunately no longer work. I hope to get the repository properly updated at some point, but for now, I'll just describe a workaround here. Regrets for the inconvenience.

The problem is that Espalier is using a version of Meteor that is too old to successfully download packages from https://warehouse.meteor.com/ since the Let's Encrypt root certificate expiration in September 2021. As mentioned in the documentation, one possible workaround is to pass the environment variable NODE_TLS_REJECT_UNAUTHORIZED=0 to the Meteor tool, but of course, that might allow a MITM attacker to replace the packages with malicious code that would then be automatically run on your system (I haven't researched whether Meteor has additional integrity protection mechanisms that would prevent this). Instead, I suggest letting the Meteor tool download the packages via an HTTP proxy that uses a newer TLS client that can handle the certificate expiration.

One possible method in detail: Create a temporary directory outside the Espalier source tree (so the Meteor tool will use the latest version of Meteor by default) and start the following command in that directory:

meteor npx http-server --port 3023 --proxy https://warehouse.meteor.com/

Then run the following in the Espalier directory to download the Meteor packages:

METEOR_WAREHOUSE_URLBASE=http://localhost:3023 meteor --prepare-app

Assuming this succeeds, you can quit the proxy and use the normal meteor command from that point on, unless you switch to a different Espalier version with different dependencies, in which case you'll need to repeat this process.

Acknowledgments

This project was supported by a grant from Wistron Corporation as part of a collaboration between Wistron and MIT's Computer Science and Artificial Intelligence Laboratory. This project also received partial support from the National Science Foundation under Grant No. CCF-1438982, and Matt McCutchen's work was funded in part by an NSF Graduate Research Fellowship.