Dyalog ’22 Day 1: Welcome to Sunny Olhão!

Arriving to meet the warm sea air is already a refreshing change as this year’s long, hot summer started to close in the weeks leading up to this year’s user meeting. Of course, for some with less far to travel, the warmth is a familiar comfort.

Apartments at the Real Marina in Olhão, Portugal

Apartments at the Real Marina in Olhão, Portugal

As delegates arrived on Sunday night we were treated to a lovely surprise – a birthday cake to celebrate Tony Corso’s birthday! Happy Birthday, Tony!

Today we kicked off the user meeting with workshops, giving delegates hands-on experience with a range of Dyalog offerings for application development.

Attendees of Rich Park and Rodrigo Girão Serrão’s language workshops explored different ways to express ideas in APL using both tried and true idioms in the morning and newer language features in the afternoon – they compared expressiveness and performance implications in difference circumstances.

Morten and Josh got participants up to speed with using and maintaining code which lives outside of your workspace in text files, while Richard Smith and Bjørn Christensen showed how to manage and interact with data from outside of the workspace.

Brian Becker guided delegates into the world of SAAS (Software As A Service). It is encouraging to see how straightforward having APL code talk to the outside world can be. As usual, however, the complexities reveal themselves as you delve deeper in to specific use cases and circumstances.

Morten presents a Dyalog workshop

Morten and Josh show how to store source code as text files using Link

Without a doubt, getting to meet our users face to face has already proved to be the greatest enjoyment of the meeting so far. Some familiar faces long since last seen, and others only over a screen. Discussing ideas face to face still retains a value impossible to quantify.

Overall a fantastic beginning to the week and we cannot wait to experience the rest of this week!

APL Seeds ’22: Tuesday 29 March

On Tuesday 29 March we hosted APL Seeds ’22, the second annual online event for new and prospective users of APL (although everyone was welcome). Once again we were delighted to see that the majority of registrants had little to no APL experience; it feels like we get the chance to offer that same sense of discovery we felt when first learning about the language.

APL Seeds ’22 began with an introduction from Dyalog Ltd’s managing director, Gitte Christensen, in which she described her experiences of seeing APL enable people who had real problems to solve and showed some of what Dyalog provides in terms of the tools and interfaces that people might expect from a modern software development stack. Gitte explained the new Basic Licence, which is another step forward in Dyalog Ltd’s aim to bring APL to a wider audience. The Basic Licence allows non-commercial distribution of Dyalog along with APL-based solutions under the terms of the Royalty-Based Run-Time Licence, which will apply as the default run-time licence (see Prices and Licences for more information on Basic Licences). She also described some customer use cases, some of which might surprise newcomers to this language. Rich Park finished the introduction by pointing out where you can find more APL content, especially if you’re just getting started. For example, our tips for beginners includes things that might not be obvious when you first start the interpreter or read introductory books. The video description for the recording of this presentation contains many useful links!

APL is an executable notation and tool of thought which enables people with good ideas to bring them to life with computers

Gitte opens the event with her thoughts on “What is APL?”, including the short expression 1 2 3 + 4 5 6 that started the journey which eventually brought her to Dyalog.

Rich then presented a basic introduction to APL, showing the benefits of a symbolic notation for programming as well as demonstrating how to put together simple building blocks to build a function. After introducing the basic syntax, right-to-left precedence, and the generality of APL operators, along with a handful of symbols including the famous outer product (⍺∘.F⍵) and array indexing (⍺[⍵]), he walked through constructing a function that visualised the probability distribution for sums of rolling two N-sided dice.

A screenshot of an APL function to do some simple statistical computation and display the output using characters in the APL REPL

The Dist function uses just a handful of APL constructs to create a visualisation of a simple statistical distribution.

Stefan Kruger, author of the online book Learning APL, took us on an exploration of bioinformatics problems – a popular topic for the annual APL Problem Solving Competition. He described what a “k-mer” is (a chunk of DNA of a particular length), and compared different techniques for cutting up a text vector to isolate them from a DNA string, including a windowed-reduction (⍺F/⍵) and its generalised cousin, the stencil operator ((⍺⍺⌺⍵⍵)⍵), and our old friends the outer product and array indexing. Finally, Stefan looked at three approaches to doing some simple statistics in the Rosalind challenge “Computing GC Content“. To our delight, a new user who was in attendance commented that they learned new expressions and idioms that they had not seen before.

Stefan compares 3 functions to split a string into lengh-4 substrings

Stefan compares 3 functions to split a string into lengh-4 substrings

Andrew Sengul presented a more involved example, April. The April APL Compiler is a new entry in the APL field, compiling a subset of APL into the Common Lisp language and allowing APL functions to easily be used within Common Lisp programs. Andrew gave a concise history comparing Lisp and APL. He then gave a small introduction to using Lisp to write a “macro” (code that generates other code) before giving a glimpse into the implementation, architecture and design of APL in Common Lisp, as well as code that combines APL and Lisp to create visualisations. He showed a visualisation that used APL to both run Conway’s Game of Life and apply convolution kernels to show the state of cells over time. We loved seeing how he has been using April in the visual art installation Bloxl (a collection of computer-controlled light-up blocks used at events for a stunning visual effect). Andrew concluded his presentation with a demonstration of April code implementing a “falling block game”, and a video of that game in action.

Andrew gives a comparative overview of the histories of the APL and Lisp programming languages

Andrew gives a comparative overview of the histories of the APL and Lisp programming languages

Finally, there was a live recording of an episode of Array Cast, a semi-weekly podcast about array languages. From the regular panel of presenters were self-proclaimed J enthusiast Bob Therriault, Kx Librarian Stephen Taylor, Dyalog tools developer and life-long APL programmer Adám Brudzewsky, and professional C++ developer and programming language fanboy Conor Hoekstra. They were joined by a very special panel of guests: speakers from the event Gitte Christensen, Rich Park, Stefan Kruger, and Andrew Sengul, and well-known APLers Aaron Hsu and Rodrigo Girão Serrão.

The discussion began with the common beginners’ questions of keyboards, typing APL and whether you really need stickers, keycaps or a whole dedicated APL keyboard to use APL. On the topic of actually learning APL, there were mentions of even more books and other resources, including YouTube channels run by some of the podcast panellists and guests. All relevant links are included in the show notes for the episode, and you can listen to the episode on arraycast.com.

Taking a more technical turn, there was discussion of the balance between code clarity and performance. Should code be clear unless absolutely performance critical? Or is it possible to have both, where the clearer encodings and approaches are also the fastest? The episoded was capped off nicely, in response to a question from the audience, with Gitte offering her perspective on how APL can help a data analyst or engineer.

In the informal meet-up after the talks, Andrew configured a simple interface to the aforementioned “falling block puzzle game”, in which participants could control the game and see their moves played out on a Bloxl wall streamed in real (if a bit delayed) time. The players tried their best but were ultimately thwarted by the interface of clicking buttons using Zoom shared controls!

To those who attended, we hope you found the event enjoyable. Relevant materials have been uploaded to the APL Seeds ’22 webpage, including links to recordings of the presentations on dyalog.tv.

Not All Heroes Wear Capes

On Friday 15 October, the SERV S&L was presented with The Queen’s Award for Voluntary Service by the Lord Lieutenant of Surrey. Geoff Streeter was one of 20 volunteer SERV S&L members who attended.

What Is SERV S&L?

SERV S&L (Service by Emergency Rider Volunteers for Surrey and London) are a charity organisation, made up entirely of volunteers, comprising motorbike riders, car drivers, controllers, and fundraisers. They transport blood products, urgent samples, medical supplies, and donated breast milk to hospitals and milk banks across Surrey & London, as well as carrying out a daily delivery of blood to the Air Ambulance service that covers Kent, Surrey, and Sussex. They support the regular delivery rounds that the NHSBT (National Health Service Blood and Transport) have in place; unlike the NHSBT, SERV S&L also operate throughout the night. All of this is provided free of charge to the NHS, releasing more money for patient care.

What Is The Queen’s Award for Voluntary Service?

QAVS (The Queen’s Award for Voluntary Service) celebrates the outstanding work of local volunteer groups across the UK. Created in 2002 for Queen Elizabeth II’s Golden Jubilee, QAVS awards shine a light on the fantastic work of voluntary groups. QAVS awards are the highest awards given to local voluntary groups in the U.K. (they are the equivalent of a personal MBE) and they are awarded for life.

Geoff’s Involvement with SERV S&L

A Personal Recollection

At the end of 1980, Paul McCann had a relation who could not get an urgent sample transported to the testing lab until the next morning. He was frustrated by this and organised a meeting to see what could be done, the result of which was that a group of advanced motor cycle trainers from a (now defunct) group called Star Rider decided to try to run a delivery service for blood/samples at night. I was not at that meeting but I heard about it from a fellow member of the Laverda Owners group; I made it to the second meeting (on 8 December 1980) and have been involved ever since. We obtained a room with a couple of bunks in a wooden building owned by MEFAS (Malden Emergency First Aid Society) and a telephone line, and started operating in early 1981.

The main distribution point for blood is located in Tooting and serves London, Kent, Surrey and Sussex (we have partner organisations in Kent, Sussex and Wessex). We do a main nightly run with typically 6 to 10 boxes down to an arranged change point for Kent and Sussex. We also partner with similar organisations across the U.K., and have occasional relay runs, for example, from Edinburgh to central London (I think that’s the longest that we’ve been involved in). More common are runs from Bristol. We typically shift 20 boxes a night and samples in the other direction and have about 8 riders/drivers on shift every night.

Financially, we get support from some Masonic Lodges and business groups. They prefer to buy bikes for us, and Citroen have given us a car (DS3) on permanent loan. We are in the process of acquiring/refurbishing a scout facility in Sutton to provide a base for the bikes/cars/van as well as for volunteers who live on the periphery of the area. We also raise funds by box waving outside supermarkets, garden centres, Brooklands, Waterloo Station, etc.

I started with the group as a biker, and used my Laverda 750, Laverda 1200 and Honda 650 Turbo to deliver blood and samples from 1981 until 1990, when I switched to car deliveries (which I continued to do until last year). I also acted as Treasurer from 2006 until 2010. I have been one of the controllers right from the start [Ed.: Controllers orchestrate the logistics of a shift; hospitals and partner groups place their orders and riders and drivers are dispatched as required – accurate scheduling and data logging are required to ensure efficient co-ordination and communication so that each run can be completed reliably], a role that has changed a lot over the last 40 years. In the early days controllers needed to be physically present with the one rider and the telephone. Then we moved to using pagers (but still needed to be present in the hut/sports centre) before everything changed with the advent of mobile phones – I now control from home. The expectation is that volunteers do one night a fortnight, but a shortage of volunteers relative to growing demand means that for a few years now I have been doing at least one shift a week.

Final Word

Congratulations Geoff, 40 years of volunteering for such a worthy cause is a fantastic achievement. All of us at Dyalog Ltd are really proud of your contribution.

To find out more about the amazing service provided by SERV S&L, including how to make a donation, visit https://servsl.org.uk/.

Welcome Karta Kooner

Karta joined Dyalog in April, and is yet to meet anybody in person although he’s been told that this is not necessarily a bad thing! After completing his doctoral degree in theoretical physics, Karta stumbled upon Dyalog and APL entirely by happenstance. Being often captivated by things that look unfamiliar to him, and having an interest in most things, it was a code golf question that was answered in a strange, yet mathematical-looking language that took him to the profile of the poster, who happened to mention they were employed by Dyalog and currently hiring. He sent an email enquiring about the opportunity and, several remote interviews later, was happy to be hired as a C/C++ developer working on the interpreter.

Karta is one of the few members of the team that knew no APL whatsoever before joining and has been very impressed by Dyalog and APL thus far; he is very much looking forward to seeing how far the language can be taken, with an eye to further developing and potentially encouraging its use in academia and other technical fields of study.

In his spare time, Karta enjoys expanding his knowledge of both scientific and technical pursuits, and tinkering around with software and hardware systems, amongst his eclectic interests. When not found reading papers or learning an unfamiliar branch of mathematics, he will be caught thinking of a new engineering project to occupy his time, or stumbling through learning a new language, or maybe just delighting in the latest vixra paper.

News and Recommendations Regarding Dyalog Versions 17.1, 18.0 and 18.1

Background

Dyalog version 18.0 contained the largest number of optimised algorithms in the history of Dyalog APL. Unfortunately, since its release, we have found that our process for review and testing of optimisations was insufficient to cope with the quantity and nature of the optimisations. Many of the new algorithms had edge cases that escaped both the existing regression tests and the new tests that were written as part of development.

In August 2020, after the issues were reported following the release of Dyalog version 18.0, we issued a caution against the use of version 18.0 in production. We added significant additional testing during September and October, and subsequently lifted the caution in November. In April 2021, a user encountered an additional defect leading to incorrect results in a rare case of membership (), and in June a crash in an extremely rare case of “where” (monadic ). Since then, our own internal testing has found a defect in interval index (dyadic ).

Current Status

We have made patches available as defects have been detected and fixed, and “Issue 4” of version 18.0, made available to all users on 22 June 2021, includes fixes for all known issues with optimised code.

Download Dyalog version 18.0 Issue 4: Commercial users  Non-commercial users

At this time, we are executing two high priority projects, one aimed at increasing safety in the short term, the other at completely eliminating problematic optimisations within a few months:

  • We have further increased our internal testing of version 18.0, to verify the correctness of the optimisations that are still in the distributed product.
  • We are holding back the release of version 18.1, which was originally scheduled for release this month, while we remove most of the optimisations that went into version 18.0. We hope to release version 18.1 before the end of the 3rd Quarter of 2021, but will not let it out the door until we have complete confidence in this new version.

Recommendations

Dyalog Ltd recommends that users of Dyalog APL take the following action:

  • If you are using version 17.1 or earlier, plan to skip version 18.0 and upgrade directly to version 18.1 when it becomes available.
  • If you are using version 18.0, apply the latest patches or reinstall using Issue 4 when it becomes available. Continue to monitor our DSS e-mail broadcasts and apply patches quickly if we should find and fix any further issues that you think might impact your application. When version 18.1 becomes available, plan to upgrade to it at your earliest convenience.

Support for Versions 17.1 and 18.0

Considering this extraordinary situation, we have decided to extend support for version 17.1 for one additional release cycle. In other words, version 17.1 will be supported until we release the 4th subsequent version of Dyalog APL. Note that we are about to provide an updated version 17.1 installer (“Issue 3”) which collects all updates to version 17.1 since the original release.

Version 18.0 will be supported for the normal number of cycles, that is, until we release the 3rd following version. However, Dyalog Ltd recommends upgrading to version 18.1 as soon as it becomes available and you are able to schedule the upgrade.

Non-Commercial Versions

Given the nature of these defects, it is our intention to update our non-commercial distributions of version 18.0, although this process may lag behind the distribution of patches to clients paying for support by some weeks.

Conclusion

We apologise for the significant inconvenience that we know this is causing for some of you and thank you for your patience. We have made significant changes to our internal procedures regarding risk assessment, verification, and testing requirements for changes to existing primitive functions, to avoid a recurrence.

We do expect to re-apply those optimisations that we believe provide significant performance benefits to justify the risk of making changes over the next few releases, using new processes.

Sincerely,

Morten Kromberg
CTO, Dyalog Ltd.

APL Seeds ’21: Wednesday 31 March

Last Wednesday we hosted APL Seeds ’21, an event for those just starting their APL journey. Although we knew we had a good programme with some exceptional presenters in place, we very quickly had to increase our Zoom webinar limit to accommodate the 287 people who registered to attend! We were surprised and excited by the demographic, spanning 32 countries and with the vast majority having only basic or no APL experience.






The meeting started with a brief introduction from Dyalog’s Managing Director, Gitte Christensen, who shared her initial “Eureka” APL moment and gave some examples of situations in which APL is used today. Richard Park then took us on a whirlwind tour of APL’s past (including the very cool 1975 APL demonstration!) before demystifying the “beautiful squiggles” that define APL and introducing the modern resources that are available for learning APL (for a summary of these see the suggestions for learning resources available on our website).

The main presentations began with Rodrigo Girão Serrão giving a basic introduction to APL functions and syntax. Using the example of manually justifying text, he showed just how natural it is to process data in arrays by combining a few functions and operators. His initial exploration using a small snippet of text worked instantaneously and without issue on a whole book. After seeing this, hopefully you’ll feel an urge to learn some more – either because you got hooked (like Rodrigo did!) or simply because you want to learn how to think in an array-oriented way, which is very relevant in many situations today, such as when working with GPUs.

Martin Janiczek used a real-life example from the market insights and consumer trends company that he works for (GWI). Despite being a self-described “APL baby”, having learned APL for only around a month, he was able to get to grips with the tree structures that he wanted to use, and talked about how learning APL led him to change his overall approach to the problem. He achieved a highly-performant working prototype in two weeks and with only 172 lines of code, despite starting from the position of a complete APL beginner. Although ultimately his APL model was not taken into production, it inspired a complete rethink and new approach in the eventual product.

Conor Hoekstra (NVIDIA) describes himself as “not an APLer but a big fan”, and his enthusiasm is obvious and contagious! His explorations in APL are YouTube famous, and here again he deftly shows how APL can be used as a tool of thought to explore problems from many different angles with relative ease. He went through multiple different solutions to writing an All Equals function (is every element in a list the same as every other element?), playing with different primitives and comparing the performance of the solutions.

The final presentation came from Tomas Gustafsson, creator of the stunning Stormwind boating simulator. Tomas introduced the technology behind the 3D engine that he uses for his simulator, explaining the code that makes it all happen, before walking us through creating some simple 3-D shapes (a rotating triangle and an icosahedron) and the pitfalls that this entails. From the comments we know he seemed to inspire several members of the audience to want to know more… so for those of you that do, his code examples will soon be available from the APL Seeds web page.

We hope everyone found the event useful and enjoyable (the feedback seems to indicate that you did – thank you!). Relevant materials have started to be uploaded to the APL Seeds ’21 webpage – this page also includes links to recordings of the presentations, which are all on dyalog.tv: