Dyalog ’19 Videos: Week 4

Welcome to week 4 of the Dyalog ’19 recordings! This week, Tommy Johannesen of Jersie Data tells us about an interesting application that he has written in Dyalog APL. In the Copenhagen area, many school children are “fed with APL” in the sense that Tommy’s APL system connects the parents of ten thousand hungry children to about 60 vendors of school lunches. The service currently handles about 100,000 users with a system that runs under IIS and uses Dyalog as an ASP.NET implementation language. Each year 10,000 children leave after 10 years at school and another 10,000 enter grade 0; over the 30 years that Tommy has been in business, he has served more than a quarter of a million users.

Tommy Johannesen of Jersie Data

Tommy Johannesen of Jersie Data ApS

Peak time is Sunday evening, when parents and children log on to order meals for the coming week. Tommy tells the story of how he has been struggling with a memory leak that caused some user requests to fail when loads grew large; obviously not a good thing! Since upgrading to Dyalog version 17.0 he has been chasing this problem together with John Daintree – it was not completely solved at the time Tommy was at Dyalog ’19, but I am happy to be able to report that we believe it finally bit the dust in October!

The other two recordings in week 4 are centred around the topic of source code management. First, Adám Brudzewsky and I introduce the tool called “Link” which is included with version 17.1. Link creates a link (hence the name) between each function (or other “code object”) in an active workspace and a corresponding text file. The namespace structure of the workspace corresponds to the directory structure that the workspace is linked to. Changes made to either side of such a link – to code in the workspace or to files outside it – are immediately reflected on the other side.

Paul Mansour of the Carlisle Group

Paul Mansour demonstrates Git integration with AcreTools

The intention of a tool like Link is to enable the use of source code management systems like SVN or Git to manage source in the form of text files. These source code management systems allow the implementation of many different “workflows”, defined by parameters like how branches are used, how frequently merges are done, and how conflicts are handled. Git allows a lot of freedom, which many small teams (and that includes one-man-bands 😊) don’t need. Immediately following the talk on Link, Paul Mansour took the stage at Dyalog ’19 to talk about tools that he has developed to implement a simple but effective Git workflow, suitable for many APL projects.

Summary of this week’s videos:

Dyalog ’19 Videos: Week 3

Stig Nielsen of SimCorp

Stig Nielsen of SimCorp

In the third week of Dyalog ’19 recordings, Stig Nielsen from SimCorp A/S talks about recent work to turn a very large body of APL code, originally designed to work well on workstations, into a 3-tier solution. This requires turning everything “inside out”, making the application a service that can be called from cloud-based components. Fortunately, the parts of the application that have been implemented in APL have been using a model-driven approach to user interface specification for decades. This makes the task a lot simpler compared to some of the code written in other languages (which needs a complete rewrite). Stig explains both the obvious solution and the one that might actually work.

Brian Becker shows us the HTMLRenderer

Brian Becker shows us the HTMLRenderer

The other two talks that we are releasing this week are also related to producing portable user interfaces. The HTMLRenderer is a component that is integrated with Dyalog APL under Microsoft Windows, GNU Linux and Apple macOS. It allows APL applications to use HTML/JavaScript to produce Desktop applications that behave identically on all these platforms – in much the same way that NodeJS allows. Brian Becker introduces new features of the HTMLRenderer in version 17.1, and a number of tools that allow you to build user interfaces that run not only on the HTMLRenderer, but also as a Web Server.

Josh David demonstrates his Easy GUI

Josh David demonstrates his Easy GUI

Josh David follows up with a talk about “EasyGUI”, a tool that he built as his final year project at Scranton University. EasyGUI is built upon the HTMLRenderer, and allows applications to have a very simple UI for collecting input from the user and for displaying progress bars, simple reports or graphics. It does this without requiring that the user learn any HTML, JavaScript or CSS – it is all generated by a set of very simple functions.

Summary of this week’s videos:

See you next week for the next three recordings from Dyalog ’19.

Dyalog ’19 Videos: Week 2

Welcome to the second week of recordings from Dyalog ’19 in Elsinore! This week we are featuring three presentations about work that is either already available, or will appear in Dyalog version 18.0 in 2020.

APLcart logo

All three talks are by members of the Dyalog development team, but Adám Brudzewsky’s talk on APLcart is labelled U14 because the bulk of the work has been done in his own time. APLcart is a novel way of making it easy to find information about how to do things in APL, as opposed to finding documentation for a particular feature of the language or development environment. Although it is brand new, APLcart is already a very effective tool. I recently had the pleasure of teaching an introductory APL course where I told the students about APLcart on the first day. By the end of the week, every time the students had to solve an exercise, they headed straight for APLcart. Please check it out, try to find things, and let Adám know about anything that you can’t find so he can add it!

Richard Smith talks configuration files

One of our most important goals is to make Dyalog APL as capable – and as similar – as possible on all significant computing platforms, ideally allowing you to develop on any supported platform and deploy the solution to all others. One of the remaining hurdles is configuration of the interpreter, the development environment – and applications. This is currently done very differently from one platform to the next. Richard Smith introduces a project he is working on that aims to provide a single format for configuration files across all platforms. The choice of a file format is not an easy one, and a significant part of this recording consists of comments from the audience – including members of the Dyalog development team – voicing a variety of opinions about this.

Marshall talks tacit techniques

Finally, a delicacy for language geeks: The word tacit means “unspoken”. Tacit programming involves expressions consisting only of functions, operators and constants; the arguments are implied. For example, the expression (+⌿ ÷ ≢) is a function train, which should be read “sum divided by count” – without speaking about what the functions are applied to. The corresponding “explicit” expression would be {(+⌿⍵) ÷ ≢⍵} which mentions the argument twice. In his talk on Tacit Techniques, Marshall explains how tacit programming can be elegant and powerful, and why a set of new operators planned for Dyalog version 18.0 will be particularly useful for tacit programming. His talk includes a neat way of ranking poker hands with the over () operator, which will be available in Dyalog version 18.0.

Summary of this week’s videos:

Join us again next week for another three recordings from Dyalog ’19.

Welcome to the Dyalog ’19 Videos!

It is still true that most APL users live north of the equator, which means that at this time of year the sun is below the horizon more than half the time. It’s the perfect time to snuggle up in a warm place and watch some videos, and we’ll be offering you about three a week from now until just after the Winter Solstice, when we can start looking forward to the Spring.

As usual, we’ll be releasing videos of the vast majority of the presentations that were made at the recent Dyalog ’19 user meeting, which was held in Elsinore, Denmark in September. If you would like to get a feeling for what is coming, take a look at the blogs from the meeting itself, starting with this one.

In accordance with tradition, we’re opening with the three keynote presentations by Dyalog’s CEO Gitte Christensen, CTO Morten Kromberg, and Chief Architect John Daintree.

Gitte welcomes delegates to Dyalog '19

2019 is the first “Year of the Hammer”: after seven years of Norse wyrms and seven years of Viking ships, we were now entering the era of seven hammer-inspired logos. As Gitte explains in her talk, we are celebrating the first year under Thor’s Hammer by making Dyalog APL freely available for non-commercial use – without requiring registration – under Microsoft Windows, Apple macOS and GNU/Linux (including a collection of public Docker images). The intention is to make APL much more easily accessible for experiments – especially in the cloud!

Gitte’s talk also contains sombre tones, remembering that we lost John Scholes in February as well as Harriett Neville, who had registered to attend Dyalog ’19 but passed away most unexpectedly just before the meeting.

After last year’s Technical Road Map, which was almost entirely a live demonstration of using APL with modern development tools like Git, VS Code and Docker, I decided to play it safe this year and do no demos at all in my keynote. Instead, I concentrated on explaining some of our thoughts about making Dyalog APL easier to discover, learn and integrate into modern frameworks and development processes – and making applications written in APL easier to deploy and maintain. As a result, despite the world premiere of our new Webinar Jingle, composed by Stefano Lanzavecchia (short and long versions are available), I probably shocked the audience by leaving three minutes at the end for questions!

 

The title of John’s talk was “Cor(e) Blimey!”. The Cor(e) is of course a reference to Microsoft’s “.NET Core” but if English is not your first language, the title of John’s talk may need a little explanation. “Cor blimey” is an exclamation of surprise, a euphemism derived from “God Blind Me”. In this talk, John explains how Dyalog is poised to provide a bridge to Microsoft’s new portable, open source version of .NET. Scheduled for release with Dyalog version 18.0 next year, this will provide APL users with access to a vast collection of libraries under Linux and macOS, in addition to Windows.

Summary of this week’s videos:

I hope you enjoy these presentations. Join us again next week for another three recordings from Dyalog ’19!

A Blustery Spring

Dyalog version 17.1 will be released soon, with the HTMLRenderer working under Windows, macOS and GNU/Linux, the “Link” system providing infrastructure for connecting APL to source code management systems, pre-built Docker containers with Dyalog APL for Linux installed and many other enhancements that simplify the installation and maintenance of systems based on Dyalog APL.

We’ll be writing much more about version 17.1 soon, and next year’s 18.0 release in due course. The main purpose of this blog entry is to let you know about new members of the Dyalog team and, unfortunately, a couple of departures as well.

Departures


In February, John Scholes passed away. Together with Geoff Streeter, John was one of the original implementors of Dyalog APL in 1982-1983, a cornerstone of all aspects of the Dyalog language and business, and one of the pillars of the APL community. Many members of the community have paid tribute to our Genius, Gentleman and Mischievous Schoolboy at http://johnscholes.rip.

At the end of May 2019, Jay Foad is leaving Dyalog to return to his first love (as a software developer) and become a proper compiler geek again, after nearly a decade of helping move Dyalog APL forward and, for the last three years, helping to “herd the cats” as CTO. We will sorely miss Jay’s technical excellence but understand the desire to hit the sweet skill spot when the opportunity arises, and we wish him good fortune in that pursuit! You can read Jay’s farewell blog post here.

Jay’s management responsibilities will be shared between Richard Smith, our Development Manager and myself; I will be re-assuming the role of CTO until further notice.

New Faces in 2019

The good news is that we will welcome several new people to Dyalog in 2019 – new hands to write code in APL, to work on the APL interpreter, and to write documentation and training materials to help new and old users get their work done more effectively.

APL Consultants

In response to client requests and to help new clients get started writing their first APL systems, we are creating a consulting group in the USA. To date, we have recruited two members for this team: Nathan Rogers joined the team at the end of April and is based in Denver, Colorado, and Josh David starts work for Dyalog in early June (as soon as he graduates) and will be based in New Jersey. If you think you have heard of Josh before, that is probably because he was a winner of the Dyalog Problem-Solving Contest in 2016 (https://www.dyalog.com/news/112/420/2016-APL-Programming-Contest-Winners.htm) – and a runner up in 2015. Nathan found us thanks to Adam Brudzewsky’s work on Stack Exchange: https://chat.stackexchange.com/rooms/52405/the-apl-orchard. You can reach them both using e-addresses in the form firstname at dyalog.com.

When members of the consulting team are not working for clients, the intention is that they will be members of the APL Tools Group at Dyalog, working on new tools for APL application development and helping create test suites for Dyalog APL. They will also support Richard Park, who joined us late in 2018, to work on the creation of training materials and tutorials for new users.

Once we have a better idea of the demand for consulting in North America, we expect to grow the team. Please let us know if you could use hired APL hands – in any territory! If we don’t have the resources ourselves, we may be able to find someone else.

Programming Language Implementors

Nathan comes to us with experience from a broad set of tools and programming languages. In addition to writing tools in APL, he will be a part-time member of the core development team, working on the APL interpreter and its interfaces in C, C#, JavaScript, Python and other languages. However, he won’t spend enough time on this to make up for the loss of Jay, who (like most managers at Dyalog) spent a significant amount of his time writing code.

Therefore, as described at https://www.dyalog.com/careers.htm, we are recruiting at least one C / C++ programmer to help us grow the core team.

A Busy – and Exciting Time

2019 is looking like an extremely busy year, with significant growth at Dyalog. As usual, our plan is to bring all the new (and old) hands to the Dyalog user meeting, which will be held in Elsinore, Denmark this year – September 8th to 12th. Details of the programme will soon start to appear at https://www.dyalog.com/user-meetings/dyalog19.htm. If you would like to present an APL-related experience to the user community, make proposals for new features of Dyalog products or suggest topics that you would like Dyalog to speak about at the user meeting, then please let us know as soon as possible!

Goodbye

At the end of May 2019 I am leaving Dyalog, so it seems like a good time to reflect on my time here and what I’ve learned from APL and the APL community.

When I joined Dyalog in 2010 I knew nothing about APL, so there was a really steep learning curve as I got to grips with both the language and its implementation. I was using some of my previous experience with compilers to improve the performance of the implementation, and thinking about ways to compile APL. This is a tough problem, and one that many people have worked on over the years (see for example Timothy Budd’s 1988 book An APL Compiler). My own ideas have shifted as I’ve gained more experience with APL and the way it is used. At first I thought “writing a compiler” was an obvious thing to do; now I think that hybrid compiler/interpreter techniques are much more promising, and Dyalog’s recent experiments with deferred execution and thunks are a good step in that direction.

At the same time, there was a lot of excitement around the APL language itself. Dyalog was working on APL#, a new .NET-based APL dialect (sadly abandoned as Microsoft’s own commitment to .NET waned). And Dyalog APL itself was starting to borrow more language features from the SharpAPL/J branch of the family tree, starting with the Rank operator and continuing over many years. This prompted me to delve more into the history of APL, to try to understand some of the fundamental differences between different implementations, so that we could reconcile those differences in Dyalog APL and provide, as far as possible, the best of both worlds. I think we’ve done pretty well in that, as evidenced by the fact that many APLers are happily using Rank, Key, function trains et al in an APL2-based language, something that seemed unthinkable a decade ago.

One of the most gratifying developments in the time I’ve been working with APL is the rapid growth of new APL implementations, open source projects and grass-roots enthusiasm. In particular, the open source movement has made it much easier for anyone with a good idea about language design to implement it, and share it with the world. We’ve seen a wide variety of new APLs and APL-inspired languages popping up over the years, ranging from full-featured to highly experimental, including but not limited to (in roughly the order I remember hearing about them): ELI, ngn/apl, GNU APL, Ivy, Aprildzaima/APL and APL\iv.

And speaking of new APLs, of course there is Co-dfns, a compiled APL implementation that tries to solve another tough problem: harnessing the power of GPUs and other massively parallel hardware in a way that makes it accessible to the end user. This is something that many people are trying to do, in a wide variety of languages, but as far as I can tell no-one has quite succeeded yet. The state of the art is still that, in order to get good performance, you need quite a lot of mechanical sympathy for the underlying hardware. But Co-dfns has come a long way, and if any language is well-suited to run on parallel array processors then surely it is APL!

This brings me on neatly to my next job: I’ll be working on compilers for GPUs, the parallel computers that render 3D graphics. They are closely related to their “general purpose” cousins the GPGPUs, which are used for pure number crunching, and to so-called tensor processing units (TPUs) that simulate neural networks for use in machine learning and artificial intelligence. “Tensor” here just means an array of arbitrary rank, or as we would say: an array. For programming TPUs there is a Python-based framework called TensorFlow. But, look closely at the APIs for some of the core TensorFlow libraries, and you’ll see operations like reshape, reverse and transpose, which are eerily similar to their APL equivalents. There truly is nothing new under the sun!

With fond regards to all APLers,
Jay.