We have returned to the (very windy this time!) shores of Elsinore in Denmark for our annual user meeting, Dyalog ’23!

Today saw the usual range workshops in which everybody could get involved learning the intricacies of a range of topics with the support of experts from Team Dyalog and friends.

In the morning, users in part 1 of the Projects and Packages workshop started consuming APL packages using the Tatin package manager. While many existing users have gone decades without these very convenient additions, we see packages as an important part of APL’s future.

In the Tacit Techniques workshop, participants learned the basics of function composition operators and function train syntax. They practised spotting patterns to make explicit code more succinct, and even those who only started using APL earlier this year had great success converting explicit functions into those that do not mention their arguments.

Meanwhile, users who wanted a challenge tackled an advanced workshop focussing on common pitfalls when writing code that performs well.

Projects and Packages workshop with Morten Kromberg, Kai Jaeger, and Gilgamesh Athoraya

In the afternoon, we hit the rate limit on the GitHub API during the session on web services, as presumably everyone on the venue’s Wi-Fi network had the same I.P. address! Fortunately, after each user successfully configured a web service on their own machine, everyone could experiment as much as they wanted to.

In part 2 of the Projects and Packages workshop, participants managed to publish their own APL packages to the Tatin test registry for others to use. Now that they’ve had a taste, we hope to see more become available on the main Tatin registry in the future!

The development masterclass took participants through some basic and some lesser-known features of Dyalog’s Integrated Development Environments. Some newer APLers said that they learned a lot, and even more experienced users said they had benefitted and would take the materials back to their organisations to share with others.

Adám Brudzewsky and Stefan Kruger host a masterclass on the features of the IDEs

Conversation over dinner included catching up with long-time friends and making new ones, stories of onboarding new APLers (some of them in attendance), and hearing about personal and professional achievements of the last few years. There’s excitement in the air this evening, and we can’t wait to start a fantastic few days of presentations tomorrow!

Follow

Links to recordings from this day of the user meeting are at the bottom of this post.

To begin the final day of the user meeting, Aaron Hsu presented some strategies and ways of thinking about processing data that come from text. While his specific focus is parsing APL code for the co-dfns compiler, he believes that a parallel data-flow approach will be useful across domains, for example, when processing JSON data, which is common in modern web communication.

Aaron talks about parsing data from text

Adám Brudzewsky, despite not being in the room, captivated us with a supremely convincing argument for why three primitives could be considered to be filling in “core language” gaps: the “Select” function (⍺⊇⍵) for elegant indexing; the “Depth” operator (F⍥k) as analogous to rank (F⍤k) for nested arrays; and the “Behind” (F⍛G) operator complementing other function compositions “Beside” (F∘G), “Atop” (F⍤G) and “Over” (F⍥G) to make tacit programming an even better extension of APL expressivity.

Adám proposes a new “behind” function composition operator

It was exciting to see a return for APL to its birthplace in academia. Jesús Galán López of the Technical Univerity of Delft told us his plan to use APL in metallurgical models and analysis of experimental data. He hopes to use Dyalog APL as a central connecting piece in education, science, engineering and research. This is one of the first projects to eventually be supported by the APL fund mentioned by Stine on Monday.

Jesús with a typical data analysis example in materials science

A living legend in APL education, Raymond P. Polivka presented his latest book aimed at high school students with no programming knowledge whatsoever. The clearly structured modules and exercises will be an invaluable resource for teachers and recruiters alike, and they have been developed with both a lifetime of experience teaching APL as well as very recent classes with real high school students. He is asking for feedback on a draft available at aplclass.com/book.

Ray Polivka introduces his new book aimed at high school students

For the last talk of the user meeting, Rich Park took us on a whirlwind tour of the media and training materials being developed at the moment. He also commented on some of the work that he believes is needed to tie these resources together and make them more discoverable, and what might be done to improve the experience for new users trying to find things out for themselves.

Rich Park talks about media, training and outreach at Dyalog

In the afternoon we ended the user meeting with some workshops. We got to know about performance considerations in the context of the Dyalog interpreter with Aaron Hsu and Karter Kooner. There was a chance to get hands-on experience using one of the asynchronous, concurrent programming methods in Dyalog: Futures and Isolates with Morten Kromberg and John Daintree. Josh David and Brian Becker taught us about sharing APL tools by creating and publishing packages using Tatin.

It has truly been an incredible experience returning to in-person user meetings. The sense of community felt in Olhão this year was far beyond what most people predicted, not just in light of separation over the last two years, but also in looking forward to an exciting future for APL and Dyalog.

Today’s presentations (links to recordings will be added as they become available):

• D14: Text Processing in APL – Aaron Hsu
• D15: Filling the Core Language Gaps – Adám Brudzewsky
• U16: Dyalog and Academia – Jesús Galán López
• U17: What – Another APL Book? – Ray Polivka
• D16: Growing APLers – Rich Park

Follow

Links to recordings from this day of the user meeting are at the bottom of this post.

The third day of presentations kicked off with Brian Becker running us through the gauntlet of setting up and deploying web services. Initial setup with Jarvis and Docker containers seems like an absolute breeze. However, the later stages configuring cloud services can be more fiddly.

Brian Becker talks about creating and deploying web services.

Stephen Mansour of Misericordia University then gave us the hot tips for producing hot tubs. His new system TAMPA (Taming Mathematical Programming in APL) can be used to optimise some decision (e.g. how many hot tubs to produce) for some objective function (e.g. maximise profit) according to some constraints (e.g. resources available). The use of APL in TAMPA makes a near 1-to-1 translation of linear programming expressions into executable code.

Stephen Mansour explains the TAMPA mathematical programming framework.

We then got to hear some strong but convincing opinions about graphical user interfaces from Norbert Jurkiewicz, who told us about how The Carlisle Group has been incrementally integrating the HTMLRenderer and “the triad” of HTML, CSS and JavaScript into their systems. He championed the benefit of hiring external developers, in addition to the portability of using web-technologies, for graphical front ends.

Norbert Jurkiewicz gives his views on using the web stack for front end development.

Neither Gitte Christensen nor Brian Becker are shy about saying that some of their favourite parts of every user meeting are the competition presentations. Luckily for us, both this year’s student winner and professional winner came to share their experiences about participating.

Professional winner Michael Higginson had actually been a kdb+ and q programmer for many years before recently deciding to expand his horizons with APL. He gave a fantastic breakdown of his thought process in solving both an easier problem which he found built his confidence, and then the notorious problem 6 on interpreting human-readable dates and times.

Michael Higginson takes us through his array programming journey.

The audience could definitely empathise with all of the reasons given by student winner Tzu-Ching Lee as to why he likes APL: the glyphs; the concise syntax; operators; and algorithms as primitives. Alongside his excellent walkthroughs of two of his solutions, he had brilliant ideas for extending the problem description for the Base85 encoding/decoding problem once he noticed additional symmetry which could be expressed in his solutions.

Student winner Tzu-Ching Lee presents his winning solutions to the APL Problem Solving Competition.

In the afternoon, we took a coach about an hour away to the Quinta dos Vales vineyards and winery. We were treated to a tour of the winery, learning about the fermentation process; what goes into deciding whether to make single grape or blended wines; and the use of wooden barrels to imbibe additional flavour. Afterwards, we were split into teams and challenged to make our best and favourite blends of wines from three grapes. According to the judges, a majority Cabernet Sauvignon, with about a third Aragonês and just ten percent Touriga Nacional makes for the most delicious blend of tannins and spices. Later that evening, we enjoyed a delicious Portuguese churrasco – or barbeque.

Delegates enjoy the afternoon sun at Quinta dos Vales winery in the Algarve, Portugal.

Congratulations to the winners of the APL Problem Solving Competition, and congratulations also to the winners of the wine blending competition!

Today’s presentations (links to recordings will be added as they become available):

• D12: Simplifying Secure, Scalable Web Services – Brian Becker
• U12: TAMPA – Taming Mathematical Programming in APL – Stephen Mansour
• U13: Integrating HTMLRenderer Into Existing Applications – Norbert Jurkiewicz
• The APL Problem Solving Competition – Introduction and Prize Ceremony – Brian Becker
• How I Won the APL Problem Solving Competition (Professional Entrant) – Michael Higginson
• How I Won the APL Problem Solving Competition – Tzu-Ching Lee

Follow

Links to recordings from this day of the user meeting are at the bottom of this post.

Automation, architecture and performance were throughlines of the second day of presentations. Lars Stampe Villadsen from SimCorp A/S provided some advice on how to write tests and gave a live demonstration of a small testing framework together with continuous integration tools which run the test suite every time a change is committed to the project – useful for when we forget to run some tests locally.

Lars Stampe Villadsen talks about testing.

Norbert Jurkiewicz presented his “10,000 ft overview” of automation processes used by The Carlisle Group and described some of the complexities of building distributables according to varying requirements for different customers. He discussed some of the cost considerations to be made when using Amazon Web Services (or any cloud computing service) to leverage lots of computing power to do builds quickly.

Norbert Jurkiewicz discusses automation architecture.

This was followed by Michael Baas with a similar issue viewed from a different angle. He talked about using the ]DTest testing framework to produce code coverage reports – showing which lines of code had actually been executed when running a test suite – and also automating testing across different platforms and versions of the Dyalog interpreter.

Michael Baas shows us the ]DTest framework

Changing tack, we heard the story of getting to grips with semi-global variables in a multithreaded application. Elena Pavarotti of SimCorp Italiana talked about her experience trying to imagine clearly what a complex system is doing so that it is easier to reliably refactor and adapt it to talk to new external systems.

Elena Pavarotti on managing complexity in SimCorp Sofia.

Rodrigo Girão Serrão then told us how himself and Aaron Hsu had implemented a U-Net Convolutional Neural Network from scratch in APL. The comparisons between their implementation and industry standard libraries was very interesting, and then seeing the mapping from diagrams to code showed us how the APL becomes a natural way to express the data flow involved in the system.

Rodrigo Girão Serrão walks us through the U-net CNN in APL.

Delving deeper into more intellectual musings, Justin Dowdy of Semantic Arts – also known for his work on the April APL to Lisp compiler and the May bridge between Dyalog and Clojure – drew some interesting analogies between the Resource Description Framework, used for representing data relationships as ontologies, and points raised in Iverson’s paper Notation as a Tool of Thought.

Justin Dowdy compares Notation as a Tool of Thought and concepts of Data Semantics.

Juuso Haavisto is a DPhil student at the University of Oxford. He brought up some hot topics in computer science academia – static analysis, rank polymorphism and scheduling for multi-core systems – that he believes can be tackled effectively if we can learn to make the computer think a bit more like an APLer.

Juuso Haavisto with three hot topics in computer science academia.

The theme of performance continued when Vali-Matti Jantunen from Statistics Finland compared the performance of some short APL phrases across several versions of Dyalog. Of course, this can be important in an application like PxEdit which processes thousands of text files; a fraction of a second difference processing a single file can lead to several minutes across a job.

Vali-Matti Jantunen on Dyalog performance across versions.

We hope Veli-Matti will forgive us for the performance regressions found in v18.2 (although not slower than v17.1). However, for us it was no surprise, and next Karta Kooner reflected on some of the assumptions made when changes, intended to improve performance, had been implemented over the lifetime of Dyalog APL. He is looking for volunteers to run a version of the interpreter which can gather usage statistics, so please get in touch if you can do this.

Karta Kooner analyses performance in the interpreter.

Presenting the final features of his special Dyalog ’22 Conference Edition of the interpreter, John Daintree talked about the state of asynchronous programming in the interpreter now. Eventually he showed us an interface, in the from of an ⎕AWAIT system function, to unify Spawn (F&⍵) threads, .NET Tasks and Futures and Isolates.

John Daintree with some ideas about asynchronous programming.

Finally, Aaron Hsu’s Co-dfns report showed a new public API for the parser which could be useful for static analysis of existing code and better error reporting within co-dfns. The new code generator written in APL opens the door to targeting more platforms more easily in the future.

Aaron Hsu presents an update on co-dfns.

Today’s presentations (links to recordings will be added as they become available):

• U06: From “I Developed and Tested It” to “I Developed, and My CI Tested It” – Lars Stampe Villadsen
• U07: Automating Application Builds with AWS – Norbert Jurkiewicz
• D07: Test Your Code! – Michael Baas
• U08: Semi-Globals and Multi-Threading are Like Chalk and Cheese – Elena Paviotti
• D08: Implementing the Convolutional Neural Network U-Net in APL – Rodrigo Girão Serrão
• U09: APL on the Side – Justin Dowdy
• U10: Scheduling Array Operations – Juuso Haavisto
• U11: Performance of Dyalog APL – A Historical Perspective – Veli-Matti Jantunen
• D09: Performance Improvements in Set Operations – Karta Kooner
• D10: 2022 Conference Edition Part 3a – Future(s) – John Daintree
• D10: 2022 Conference Edition Part 3b – Future(s) – John Daintree
• D11: Report on Co-dfns – Aaron Hsu

Follow

Links to recordings from this day of the user meeting are at the bottom of this post.

We were welcomed to the 2nd day of the user meeting by managing director (CEO) Gitte Christensen. Everybody could appreciate the emotion and sentiment when she noted the distinct “APL Hum” apparent in the atmosphere at this year’s event.

Managing Director Gitte Christensen welcomes us to the user meeting.

In some exciting news, Stine Kromberg announced that Dyalog’s board of directors have approved an APL fund for education and science. She also made a call to action for members of the APL community to present their stories to the wider world.

Stine Kromberg announces the APL fund

Morten Kromberg’s road map provided an overview of the multitude of projects and requirements Dyalog is to address in the upcoming years. From the need for new APLers to be recruited to replace those career programmers leaving the fold, to developing tools and training resources that can help all users navigate the modern, complex world of interconnected software development and making the user experience more pleasant and consistent across different platforms.

Morten Kromberg presents the roadmap of activity at Dyalog.

Morten later returned to speak about the status of project and package managers for Dyalog. While Carlisle Group’s Dado is an excellent recommendation for those who can adopt its tools and workflow, Dyalog Ltd has employed the help of APLTeam to develop the more agnostic Tatin package manager and Cider project manager, with the hopes of providing a more flexible and extensible set of tools to meet a wider range of needs.

In a double bill of talks from John Daintree, we were shown a special 2022 Conference Edition of the Dyalog interpreter which has features stemming from both internal discussions and projects and our users’ requests and ideas. He reiterated some of the complications of input and output in Dyalog, but showed how the conference edition can more intelligently handle output from the various sources – for example error messages, quad output ⎕←message and output from system commands.

In his second talk, he demonstrated a system for piece-by-piece debugging of inline APL code and functions. We think you will agree that being able to step through the evaluation of a single line of APL has massive implications for teaching, writing and understanding APL.

John Daintree demonstrates inline “token by token” debugging.

Charles Brenner from DNA-View took us on a journey of his discovery of a method of performing numerical integrations in high dimensional spaces – using a kind of Riemann summation (analogous to the rectangle or trapezium methods) using simplexes (multidimensional generalisation of triangle and tetrahedrons).

Charles Brenner talks about numerical integration

Gilgamesh Athoraya presented a recent project by Tiamatica AB to migrate a process planning system from mainframe APL2 to Dyalog in the cloud. This large undertaking began with creating a bridge from Dyalog to APL2, and the challenge continues as code must be migrate to Dyalog while being able to fold in updates made by the APL2 team still maintaining the current software.

Gilgamesh Athoraya talks about migrating APL2 mainframe to Dyalog in the cloud.

Another APL2 conversion, Mark Wolfson of BIG provided some fascinating insights into the world of Jewelers in North America. Many of these business have managed to resist the trend of becoming parts of large conglomerates and have stayed as small, often family owned businesses. This has meant that they usually don’t have sophisticated software for managing their sales and inventories. Mark’s system uses APL to ingest data from an incredible variety of formats and provide insights to help these businesses thrive.

Mark Wolfson gives insight into the world of North American Jewellery retailers.

Unfortunately, Kimmo Linna from Finnair could not join us in person due to his obligations as a pilot. Fortunately for us, his time away from the cockpit during the pandemic led to him developing useful tools and workflows that he was able to show us remotely via video feed. He is using DuckDB to store data and Vega-lite to present it visually, all being driven from Dyalog Jupyter notebooks and connected using bridges that he has published and made available for free on GitHub.

Kimmo Linna presents his Dyalog/DuckDB/Vega-lite/Jupyter workflow

To finish the day, new recruit on the development team Peter Mikkelsen talked about a personal project from before he joined Dyalog: an APL implementation for the Plan9 research operating system. As he explained, it may be lacking performance, documentation and even users, but we could see from his approach and language design choices that he is someone who can bring great ideas to the interpreter development team.

Peter Mikkelsen shows message passing in APL9

Today’s presentations (links to recordings will be added as they become available):

• Welcome to Dyalog ’22 – Gitte Christensen and Stine Kromberg
• The Road Ahead – Morten Kromberg
• D03: 2022 Conference Edition Part 1 – Past – John Daintree
• U01: One-Time Pure Mathematician Corrupted by Exposure to APL Loses Moral Compass and Discovers, after Several Mis‑Steps, a Useful Numerical Integration Method – Charles Brenner
• U02: Lift-Off from APL2 Mainframe to Dyalog in the Cloud – Gilgamesh Athoraya
• U03: Rebuilding a Production APL Environment using Dyalog – Mark Wolfson
• D04: 2022 Conference Edition Part 2 – Present – John Daintree
• D05: The P words…Projects and Packages – Morten Kromberg
• U04: A Modern APL Workbench – Kimmo Linna
• U05: An APL to Vega-lite Interface – Kimmo Linna
• D06: APL9 from Outer Space – Peter Mikkelsen

Follow