Swat.engineering & Rascal's evolution

The tragedy of the commons

The tragedy of the commons is a well-known metaphor for a common good that is destroyed by citizens’ greed and lack of care. If all sheep of the village use the common meadow too often, the meadow will become dry and barren and will no longer feed any sheep. In a similar fashion, open-source software constitutes a gigantic software commons from which the whole world benefits: companies and citizens alike. How does Swat.engineering contribute to this software commons?

The history of Rascal

Rascal originated in 2009 at Centrum Wiskunde & Informatica (CWI) in Amsterdam, the Netherlands. Its development was funded by several research grants and driven by three core committers located at CWI and Swat.engineering: Paul Klint, Davy Landman, and Jurgen Vinju supported by 55 international committers. Together, they have produced over 20,000 commits. Rascal has experienced steady development over the years, as shown in the graph below.

Rascal is increasingly used by universities and companies alike. This growth means that the effort of maintaining Rascal’s code base and educating its users can no longer be borne by researchers alone. Rascal now faces the tragedy of the commons.

Swat.engineering’s contributions to Rascal

Swat.engineering was established in 2016 to stimulate the application and further development of Rascal in commercial projects. The availability of Rascal as a not-so-secret weapon has hugely benefited Swat.engineering in all its commercial projects.

We contributed to Rascal even during the usual struggles of starting a company. Over the years, our team has made many contributions to the Rascal commons. Here are some highlights:

• VS Code plugin & Language Server Protocol (LSP): Originally, IDE support for Rascal was based on Eclipse and was tightly coupled with the Eclipse infrastructure. Recently, we completed an implementation of LSP for Rascal and used it to build a Rascal plugin for VS Code. It exploits many of the modern IDE features that VS Code provides, such as code actions, code lenses, hover help, and fixes. These days, the Rascal plugin for VS Code is the way to go for Rascal development. Benefits: a better and more efficient development process for Rascal in a modern IDE.
• Rascal typechecker: Rascal has been designed as a statically typed language but its interpreter-based implementation did not enforce all of the language’s static typing rules. We have invested in a Rascal typechecker that is precise, helpful, and fully integrated in VS Code. Benefits: early error detection and more reliable code.
• Error recovery: Rascal continued earlier pioneering work on generalized parsing. However, one crucial piece was missing: error recovery. This has been a long standing wish for improving the Rascal user experience. We fixed this by extending the existing Rascal parser with error recovery capabilities. The benefits are numerous: (a) only the area near a syntax error is shown as an error, and the surroundings of the error are fully parsed; (b) syntax highlighting remains intact even in the presence of syntax errors (also see next point); (c) tools such as an outliner or typechecker can do some useful work on programs with syntax errors.
• Basic syntax-highlighting with TextMate: Many IDEs, including VS Code, use TextMate grammars (actually regular expressions) to define the basic highlighting of source code. We now use TextMate as the first stage of syntax highlighting unparsed source code. Once the source has been parsed, semantic highlighting takes over since it has more knowledge about the syntactic category of text parts and the best way to highlight them. Benefits: in combination with error recovery, source code is now always highlighted in a plausible manner.
• Rename refactoring: Books have been written about code refactoring, but the most important refactoring remains renaming. We have implemented a full-fledged renaming tool in VS Code. Benefits: easier renaming leads to better names and more maintainable code.
• Rascal Shell/JLine3: We are integrating the latest and greatest terminal technology in our REPLs. Benefits: all the wonders of JLine3 are now available in the Rascal Shell, such as advanced command history, syntax highlighting and error reporting even in mutli-line input, etc.
• Monitoring file changes with Java Watch: A crucial feature of an IDE is knowing which files have been changed. Changed files need further attention like parsing, type checking and the like. Unfortunately, doing this in an OS-independent and correct manner is a challenge. We have developed a Java library that aims to achieve this. Benefits: detecting file changes brings more incrementality to the Rascal IDE such as even more incremental type checking.
• Rascal deployment: improvements to the build process, Maven integration, IDE deployments and testing. Benefits: all Rascal users can take advantage of more frequent, even more stable Rascal releases, and the ability to deploy Rascal based extensions.

Key takeaways

The tragedy of the commons can be avoided if the users (primarily companies) of open-source software behave responsibly. Swat.engineering has decided to contribute to these commons as much as it can. This is gratifying, useful, fun, and … responsible. But most of all, it further strengthens Rascal’s position as the tool of choice for software renovation and DSL development.

Get in touch

Do you have an IT challenge in, for instance, software rejuvenation or DSL development? Then reach out to us. We look forward to discussing how potential applications of our Rascal expertise and tools could solve it.

Header photo by Andrea Lightfoot on Unsplash