APL Seeds ’23: Wednesday 22 March

Pie chart of APL Seeds '23 registrant prior knowledgeLast Wednesday we hosted APL Seeds ’23, the third of our free online user meetings aimed at new and prospective users of APL. In attendance was a mix of new users and the APL-curious, as well as more experienced APLers answering questions and imparting wisdom in the chat and during the post-event meetup.

Rich Park began the meeting with an introduction to the basic syntax of the language and a sample solution to a problem from last year’s APL Problem Solving Competition – from data about the schedule of an online event, and the times people joined the event, which sessions did each user attend? He demonstrated how the thought process of a solution could map neatly onto an executable APL expression. In particular, he showed how an elegant solution tends towards using primitives directly with array argument values, as opposed to abstracting the solution for one piece and explicitly looping over the collection.

Adám Brudzewsky then explored how to remember unfamiliar ideas. He presented some mnemonic devices linking the glyphs that make up the core APL language with both the concepts that they represent and their locations on the APL keyboard. Whether they are two related concepts represented by the same symbol, or if the dyadic form is the same as monadic with a default left argument, or if two unrelated functions use the same glyph, these tricks are really useful for memorising the fundamental building blocks of APL.

Screenshot of Adám Brudzewsky's presentation

A still screenshot doesn’t do justice to the wonderful animations in Adám’s presentation.

We then heard about learning APL from the perspective of a non-programmer. Richard Savenije is a project director for infrastructure projects in a construction company. After a stint of programming in C at IBM, Richard decided he did not want to pursue programming any further. When he left, he took an APL manual with him, as he had been intrigued by the glyphs. It sat untouched for 28 years, until he recently decided to start learning to solve problems with APL. He told us about the new perspectives he has gained while using APL, and parallels with the type of problem solving he does in his day-to-day life.

APL media often shows the symbols, interesting approaches to algorithms, and other ways in which APL is fascinatingly unique in the world of programming languages. However, this often leaves people wondering whether they could use this in the more pedestrian tasks of automation, data processing, and application development. The final two talks of the day addressed the more practical use of APL.
You might have heard Stefan Kruger’s name before – he wrote the excellent Learning APL book and gave a presentation at APL Seeds ’22. He acknowledged that the domain of APL is largely seen as “super tight” numerical algorithms and array transformations. This year he explained the convenience of HttpCommand and ⎕JSON for ingesting data from external sources, and explored the creation of user commands to provide a convenient interface for performing repetitive tasks. Stefan used these to rewrite a substantial collection of (mostly Python) utility scripts as APL functions and user commands, ultimately shrinking the code by a factor of 10!

Screenshot of Stefan Kruger's presentation

Stefan shows us how to construct user commands to conveniently perform repetitive tasks.

In the final presentation of the day, Josh David answered the question of “what does it look like when my APL Seeds come to fruition?” He began with a demonstration of some analysis of water potability data that turned out to look dangerous – or fake. Throughout his demonstration, we got a feeling for how to rapidly inspect and transform data to gain insights. He included plenty of very useful tips for new APLers, such as how to stop the stream of text into the session when you execute an expression that produces long output (Action > Interrupt in the Session menu…there are more tips like this on our website). Finally, he showed some full-stack graphical applications – his SQL-Browser and The Carlisle Group’s FlipDB – with examples of database connectivity, querying and analysis.

After the talks, there was an informal online meetup. This included lively discussion about many topics, including language design and terminology. We also discussed questions about teaching tacit programming, and heard perspectives from some APL beginners – if someone is learning the concept of functions for the first time, then composition of functions is probably not appropriate at that point.

We hope we were able to spark interest, give some clarity to people’s image of APL, and inspire you to start or continue learning and using APL. Relevant materials have been uploaded to the APL Seeds ’23 webpage, including links to recordings of the presentations on dyalog.tv.

Welcome Peter Mikkelsen

Peter joined Dyalog Ltd in mid-July 2022, shortly after finishing his bachelor’s degree in Computer Science from Aalborg University in Denmark. He didn’t originally plan to look for a job so soon, but when he saw that Dyalog Ltd had an opening involving three of his favorite things (C programming, interpreters, and APL) it was too good an opportunity to let pass and he was offered the job after a couple of online calls and interviews. Peter and his girlfriend had planned to move to Esbjeg in the south-west of Denmark for a while, so he thought it was great that Dyalog Ltd made it possible to work from home. He joined just in time to be able to participate in Dyalog ’22, which meant that he met lots of customers, which he considered a great experience.

His interest in APL started while he was at university – at one point he thought to himself: “This language looks so strange and different that it might be worth learning.” After spending some time playing around with it and participating in the 2021 APL Problem Solving Competition, he started writing his own little APL interpreter, which he says must have been the point of no return for him. Even in the first few months at Dyalog Ltd, he has learned a great deal about the language and its quirks, and a lot about implementation techniques.

Peter hopes to be able to help develop the language in the years to come, and to make sure dfns get the love they deserve, as he thinks that functional programming is definitely the way to go!

Even though he is part of the “young generation” of APLers, writing long trains and doing code golfing in APL has yet to catch his interest.

When not working, Peter is likely to be found taking a long walk, or trying out something computer science related that has caught his interest. Often, the code he writes in his free time is for the Plan 9 operating system, which in his mind is so elegant and simple that it is almost a shame it has so few users.

Welcome Silas Poulson

As is often the case these days, Silas found APL through online programming language discussions – both in various forums and through videos (Aaron Hsu’s Co-dfns and John Scholes’ Game of Life demonstatration in particular) – whilst attempting to understand the more esoteric ideas and their theoretical underpinnings provided by Scheme and other functional languages. After discovering APL, Silas attempted to use it for his Data Analysis coursework, sticking primarily to a small set of primatives. However, hitting WSFULLs with the datasets required meant that APL remained just a fun system to play with, which led to an understanding of where () in terms of other primatives ( and ).

Silas saw one of Dyalog Ltd’s adverts for a C programmer; this was a language he had learnt during his exploration of the Raspberry Pi and its associated add-on boards, so he applied. He joined Dyalog Ltd in June 2022 after graduating from Royal Holloway in England with a bachelor’s degree in Computer Science, and eased his way into exploring the internal workings of the interpreter, where he now spends most of his time.

When not programmaing, Silas can be found enjoying a good book and the occasional summer hike with friends and family.

Dyalog ’22 Day 5: Looking Forward

Links to recordings from this day of the user meeting are at the bottom of this post.

To begin the final day of the user meeting, Aaron Hsu presented some strategies and ways of thinking about processing data that come from text. While his specific focus is parsing APL code for the co-dfns compiler, he believes that a parallel data-flow approach will be useful across domains, for example, when processing JSON data, which is common in modern web communication.

Aaron talks about parsing data from text

Aaron talks about parsing data from text

Adám Brudzewsky, despite not being in the room, captivated us with a supremely convincing argument for why three primitives could be considered to be filling in “core language” gaps: the “Select” function (⍺⊇⍵) for elegant indexing; the “Depth” operator (F⍥k) as analogous to rank (F⍤k) for nested arrays; and the “Behind” (F⍛G) operator complementing other function compositions “Beside” (F∘G), “Atop” (F⍤G) and “Over” (F⍥G) to make tacit programming an even better extension of APL expressivity.

Adám proposes a new "behind" function composition operator

Adám proposes a new “behind” function composition operator

It was exciting to see a return for APL to its birthplace in academia. Jesús Galán López of the Technical Univerity of Delft told us his plan to use APL in metallurgical models and analysis of experimental data. He hopes to use Dyalog APL as a central connecting piece in education, science, engineering and research. This is one of the first projects to eventually be supported by the APL fund mentioned by Stine on Monday.

Jesús with a typical data analysis example in materials science

Jesús with a typical data analysis example in materials science

A living legend in APL education, Raymond P. Polivka presented his latest book aimed at high school students with no programming knowledge whatsoever. The clearly structured modules and exercises will be an invaluable resource for teachers and recruiters alike, and they have been developed with both a lifetime of experience teaching APL as well as very recent classes with real high school students. He is asking for feedback on a draft available at aplclass.com/book.

Ray Polivka introduces his new book aimed at high school students

Ray Polivka introduces his new book aimed at high school students

For the last talk of the user meeting, Rich Park took us on a whirlwind tour of the media and training materials being developed at the moment. He also commented on some of the work that he believes is needed to tie these resources together and make them more discoverable, and what might be done to improve the experience for new users trying to find things out for themselves.

Rich Park talks about media, training and outreach at Dyalog

Rich Park talks about media, training and outreach at Dyalog

In the afternoon we ended the user meeting with some workshops. We got to know about performance considerations in the context of the Dyalog interpreter with Aaron Hsu and Karter Kooner. There was a chance to get hands-on experience using one of the asynchronous, concurrent programming methods in Dyalog: Futures and Isolates with Morten Kromberg and John Daintree. Josh David and Brian Becker taught us about sharing APL tools by creating and publishing packages using Tatin.

It has truly been an incredible experience returning to in-person user meetings. The sense of community felt in Olhão this year was far beyond what most people predicted, not just in light of separation over the last two years, but also in looking forward to an exciting future for APL and Dyalog.

Today’s presentations (links to recordings will be added as they become available):

Dyalog ’22 Day 4: Celebrations

Links to recordings from this day of the user meeting are at the bottom of this post.

The third day of presentations kicked off with Brian Becker running us through the gauntlet of setting up and deploying web services. Initial setup with Jarvis and Docker containers seems like an absolute breeze. However, the later stages configuring cloud services can be more fiddly.

Brian Becker talks about creating and deploying web services.

Brian Becker talks about creating and deploying web services.

Stephen Mansour of Misericordia University then gave us the hot tips for producing hot tubs. His new system TAMPA (Taming Mathematical Programming in APL) can be used to optimise some decision (e.g. how many hot tubs to produce) for some objective function (e.g. maximise profit) according to some constraints (e.g. resources available). The use of APL in TAMPA makes a near 1-to-1 translation of linear programming expressions into executable code.

Stephen Mansour explains the TAMPA mathematical programming framework.

Stephen Mansour explains the TAMPA mathematical programming framework.

We then got to hear some strong but convincing opinions about graphical user interfaces from Norbert Jurkiewicz, who told us about how The Carlisle Group has been incrementally integrating the HTMLRenderer and “the triad” of HTML, CSS and JavaScript into their systems. He championed the benefit of hiring external developers, in addition to the portability of using web-technologies, for graphical front ends.

Norbert Jurkiewicz gives his views on using web stack for front end development.

Norbert Jurkiewicz gives his views on using the web stack for front end development.

Neither Gitte Christensen nor Brian Becker are shy about saying that some of their favourite parts of every user meeting are the competition presentations. Luckily for us, both this year’s student winner and professional winner came to share their experiences about participating.

Professional winner Michael Higginson had actually been a kdb+ and q programmer for many years before recently deciding to expand his horizons with APL. He gave a fantastic breakdown of his thought process in solving both an easier problem which he found built his confidence, and then the notorious problem 6 on interpreting human-readable dates and times.

Michael Higginson takes us through his array programming journey.

Michael Higginson takes us through his array programming journey.

The audience could definitely empathise with all of the reasons given by student winner Tzu-Ching Lee as to why he likes APL: the glyphs; the concise syntax; operators; and algorithms as primitives. Alongside his excellent walkthroughs of two of his solutions, he had brilliant ideas for extending the problem description for the Base85 encoding/decoding problem once he noticed additional symmetry which could be expressed in his solutions.

Student winner Tzu-Ching Lee presents his winning solutions to the APL Problem Solving Competition.

Student winner Tzu-Ching Lee presents his winning solutions to the APL Problem Solving Competition.

In the afternoon, we took a coach about an hour away to the Quinta dos Vales vineyards and winery. We were treated to a tour of the winery, learning about the fermentation process; what goes into deciding whether to make single grape or blended wines; and the use of wooden barrels to imbibe additional flavour. Afterwards, we were split into teams and challenged to make our best and favourite blends of wines from three grapes. According to the judges, a majority Cabernet Sauvignon, with about a third Aragonês and just ten percent Touriga Nacional makes for the most delicious blend of tannins and spices. Later that evening, we enjoyed a delicious Portuguese churrasco – or barbeque.

Delegates enjoy the afternoon sun at Quinta dos Vales winery in the Algarve, Portugal.

Delegates enjoy the afternoon sun at Quinta dos Vales winery in the Algarve, Portugal.

Congratulations to the winners of the APL Problem Solving Competition, and congratulations also to the winners of the wine blending competition!

Today’s presentations (links to recordings will be added as they become available):

Dyalog ’22 Day 3: Automation, architecture and performance

Links to recordings from this day of the user meeting are at the bottom of this post.

Automation, architecture and performance were throughlines of the second day of presentations. Lars Stampe Villadsen from SimCorp A/S provided some advice on how to write tests and gave a live demonstration of a small testing framework together with continuous integration tools which run the test suite every time a change is committed to the project – useful for when we forget to run some tests locally.

Lars Stampe Villadsen talks about testing.

Lars Stampe Villadsen talks about testing.

Norbert Jurkiewicz presented his “10,000 ft overview” of automation processes used by The Carlisle Group and described some of the complexities of building distributables according to varying requirements for different customers. He discussed some of the cost considerations to be made when using Amazon Web Services (or any cloud computing service) to leverage lots of computing power to do builds quickly.

Norbert Jurkiewicz discusses automation architecture.

Norbert Jurkiewicz discusses automation architecture.

This was followed by Michael Baas with a similar issue viewed from a different angle. He talked about using the ]DTest testing framework to produce code coverage reports – showing which lines of code had actually been executed when running a test suite – and also automating testing across different platforms and versions of the Dyalog interpreter.

Michael Baas shows us the ]DTest framework

Michael Baas shows us the ]DTest framework

Changing tack, we heard the story of getting to grips with semi-global variables in a multithreaded application. Elena Pavarotti of SimCorp Italiana talked about her experience trying to imagine clearly what a complex system is doing so that it is easier to reliably refactor and adapt it to talk to new external systems.

Elena Pavarotti on managing complexity in SimCorp Sofia.

Elena Pavarotti on managing complexity in SimCorp Sofia.

Rodrigo Girão Serrão then told us how himself and Aaron Hsu had implemented a U-Net Convolutional Neural Network from scratch in APL. The comparisons between their implementation and industry standard libraries was very interesting, and then seeing the mapping from diagrams to code showed us how the APL becomes a natural way to express the data flow involved in the system.

Rodrigo Girão Serrão walks us through the U-net CNN in APL.

Rodrigo Girão Serrão walks us through the U-net CNN in APL.

Delving deeper into more intellectual musings, Justin Dowdy of Semantic Arts – also known for his work on the April APL to Lisp compiler and the May bridge between Dyalog and Clojure – drew some interesting analogies between the Resource Description Framework, used for representing data relationships as ontologies, and points raised in Iverson’s paper Notation as a Tool of Thought.

Justin Dowdy compares Notation as a Tool of Thought and concepts of Data Semantics.

Justin Dowdy compares Notation as a Tool of Thought and concepts of Data Semantics.

Juuso Haavisto is a DPhil student at the University of Oxford. He brought up some hot topics in computer science academia – static analysis, rank polymorphism and scheduling for multi-core systems – that he believes can be tackled effectively if we can learn to make the computer think a bit more like an APLer.

Juuso Haavisto with three hot topics in computer science academia.

Juuso Haavisto with three hot topics in computer science academia.

The theme of performance continued when Vali-Matti Jantunen from Statistics Finland compared the performance of some short APL phrases across several versions of Dyalog. Of course, this can be important in an application like PxEdit which processes thousands of text files; a fraction of a second difference processing a single file can lead to several minutes across a job.

Vali-Matti Jantunen on Dyalog performance across versions.

Vali-Matti Jantunen on Dyalog performance across versions.

We hope Veli-Matti will forgive us for the performance regressions found in v18.2 (although not slower than v17.1). However, for us it was no surprise, and next Karta Kooner reflected on some of the assumptions made when changes, intended to improve performance, had been implemented over the lifetime of Dyalog APL. He is looking for volunteers to run a version of the interpreter which can gather usage statistics, so please get in touch if you can do this.

Karta Kooner analyses performance in the interpreter.

Karta Kooner analyses performance in the interpreter.

Presenting the final features of his special Dyalog ’22 Conference Edition of the interpreter, John Daintree talked about the state of asynchronous programming in the interpreter now. Eventually he showed us an interface, in the from of an ⎕AWAIT system function, to unify Spawn (F&⍵) threads, .NET Tasks and Futures and Isolates.

John Daintree with some ideas about asynchronous programming.

John Daintree with some ideas about asynchronous programming.

Finally, Aaron Hsu’s Co-dfns report showed a new public API for the parser which could be useful for static analysis of existing code and better error reporting within co-dfns. The new code generator written in APL opens the door to targeting more platforms more easily in the future.

Aaron Hsu presents an update on co-dfns.

Aaron Hsu presents an update on co-dfns.

Today’s presentations (links to recordings will be added as they become available):