Dyalog ’19: Tuesday 10 September

2⍴⊂’APL’

Last night Roberto Minervini and his students Pietro Pio Palumbo, Gabriele Meroni and Alessandro Laselli of Liceo Scientifico GB Grassi Saronno conducted A Puzzle League – sneakily introducing us to another APL. The delegates were divided into teams who competed to solve 18 maths and logic puzzles, which could be solved both using APL and with good old pen and paper. The scoring system rewarded teams who solved puzzles that other teams did not solve, but it was soon clear that the real challenge was solving the puzzle whatsoever in many cases. The furious cognitive battle lasted long into the night but eventually the team “AdamsAPL” (Adám did not choose the name) beat “MKTeam” (Morten may well have chosen the name) to prove that, although Morten Kromberg is the CTO, he should be glad to rely on the problem solving capabilities of Dyalog’s employees.

Roberto and Pietro introduce another ‘APL’

Application Station

Marshall shows us a bit of Dyalog ‘under the hood’

This morning the second day of talks commenced. While the first talk of the day had Marshall Lochbaum melting brains with the details of utilising CPU vector processing (among other techniques) for implementing fast reductions, the majority of the day’s talks focused on various tools for developing software applications with Dyalog.

Richard Park and Michael Baas gave an update on recent developments of the statistical package TamStat. The creator, Stephen Mansour, couldn’t be with us this week as he is using TamStat to teach his class at the University of Scranton, PA.

Erik Wallace gave us a view of the wide range of functions available in his cryptographic library Mystika. His talk mentioned promising work in combination with Aaron Hsu’s co-dfns compiler to give speed ups, as well as some of his own work on algorithms and implementation. Erik also expounded on some non-cryptographic use cases such as high precision squaring and inverses.

Stig Nielsen of SimCorp

Stig Nielsen told us how SimCorp is moving Dimension to the cloud – an undertaking which requires that the business logic within its 2.5 million lines of APL code be moved from the desktop application to the server and run on multiple instances of Dyalog within a .NET process.

Asset and Liability Management has been getting more popular as the legal landscape changes and SimCorp Italiana have been swift to account for those needs. Francesco Garue gave an impression of the complexity of ALM and how SimCorp Italiana have been trying to tame their “pretty messy scheme” by, for example, removing the dependency on system-specific databases or tables.

Another asset management system was presented by Claus Madsen of FinE Analytics. Claus has been using Dyalog since version 6 so he has seen the evolution of APL user needs over the last thirty years. He showed us how he has been using .NET classes to allow his APL solution to integrate with other languages used by those he is working with, using an object oriented model to handle settings for various types of financial data.

Gooey GUI

Brian Becker shows us the HTMLRenderer

The HTMLRenderer has been developed to enable easy creation of cross-browser user interfaces using web technologies. Brian Becker drew some analogies between his granddaughter and HTMLRenderer over their respective developments, from sometimes making a mess (SYSERROR) to becoming more able to communicate over time (WebSockets). He then introduced some recent changes and additions to the way the HTMLRenderer is used in Dyalog 17.1. However, he also explained that MiServer sites would need no change to their code in order to run as a desktop application using the HRServer (HTMLRenderer Server).

Josh David has recently transferred from Dyalog user to Dyalog employee. Neatly segueing from Brian’s talk, he gave us a demonstration of the tools he has created which use the HTMLRenderer to quickly and easily create graphical elements in the Dyalog session.

Josh David demonstrates his Easy GUI

And now for something completely different

Somehow COO Andy Shiers of Dyalog keeps improving the fireside chat, and this year has reached version 5! Jokes aside, his fireside chats are an opportunity to address a few points which might not fit into the scope of other individual talks. This year Andy informed us about why it is now really important that the interpreter knows its serial number, and gave us his usual smorgasbord of tips on subjects such as: the new Windows “backtick” keyboard enhancement, resizing the language bar, the memory manager I-Beam (2000⌶) and using ⎕NINFO on directories where there are inaccessible subdirectories. In return he asked the we send him examples of APL errors which would be made clearer with better DMX messages.

Nathan Rogers creates Excel spreadsheets using Dyalog all the way from Colorado

Excel and APL: A match made in Windows – now available cross-platform. As new recruit Nathan Rogers demonstrated, since modern Excel spreadsheets are zipped XML documents using the OOXML specification, Excel spreadsheets can be created directly from APL arrays on any platform. Unfortantely Nathan could not be here in person due to an important performance of Argentine tango with his wife conflicting with the User Meeting. However, he was still able to present via video-link – what a time to be alive!

Don’t forget that tomorrow we will be live streaming the prize ceremony for the 2019 APL Problem Solving Competition on dyalog.tv from 11:00 until 12:00 (09:00 to 10:00 UTC).

Dyalog ’19: Monday 9 September

Dyalog ’19 talks begin

As usual, we began the series of User Meeting talks with a warm welcome from Managing Director Gitte Christensen. This year Gitte’s message felt somewhat spiritual as she described the lore of Thor’s hammer Mjölnir and its place in formal ceremonies. The choice of logo for this year’s User Meeting feels appropriate to the current zeitgeist: the desire to make some order from the chaos we may feel around us. You could feel a sense of hope in hard times.

Gitte welcomes us to Dyalog ’19

Hard and sad times have certainly befallen us recently and we remember two APLers who have sadly passed away this year. Harriett Neville who was supposed to attend the User Meeting, and John Scholes who left us in February. However, we also say hello to new faces at this User Meeting – Josh David and Nathan Rogers who were hired to form our new US consulting team.

In consideration of new APLers, Gitte also gave a call to arms that we should “spread the gospel” about APL. She expressed how Dyalog is making it easier to introduce people to APL by having unregistered copies of Dyalog available for non-commercial users in the near future, and also by encouraging people to share their APL tools freely online using services such as GitHub.

Morten shows us the road ahead

This was shortly followed by a road map of future Dyalog development from Technical Director Morten Kromberg. He also emphasised how Dyalog is making APL easier to find for new people, this time mentioning the APL Orchard Chat room; the Dyalog Webinars which have been running for about 2 years now; our talks in the wider programming community at events like LamdaConf and FunctionalConf; and of course the open source APL projects as Gitte had mentioned.

Morten also described how we have been working and continue to work to make APL applications easier to deploy, maintain, test and integrate with other frameworks and development processes.

There were no surprises in JD’s demonstration of the .NET Core Bridge. The functionality in terms of the .NET Framework remained, but JD showed us what it would look like if .NET “was as portable as they tell us it should be” and could be used on Windows, MacOS and Linux with the same APL code.

Marshall took us in a more technical direction, although still pointed towards the future of APL. He showed us the new operators constant , atop and over which we can look forward to in version 18.0. He also gave the suggested new nomenclature for the several types of function composition which will be available when these function-composition operators are released.

Marshall introduces new operators for Dyalog v18.0

Tommy Johannessen has been running his one-man company for some decades now, and we got to see a great user story as he demonstrated the interface to his school lunch system SkoleMad. It enables the delivery of 20,000 meals daily to 100,000 students!

Morten and Adam teamed up again to bring ]LINK to a wider audience and emphasize the importance of using text-based APL source files to modernize your APL development workflows. This is also a vital tool if you want to share your code easily with services like GitHub.

Paul Mansour demonstrates Git integration with AcreTools

The theme of modern APL development continued seamlessly as Paul Mansour of The Carlisle Group presented a Git workflow for Dyalog APL using the Acre project management system. He demonstrated using AcreTools user commands and broke down a Git workflow into something accessible even to people who are new to Git and may find it slightly daunting to use.

The co-dfns compiler is a staple of the Dyalog User Meetings at this point. Aaron Hsu’s PhD project allows APL code to run fast on GPUs, and this year he was clearly excited to show us some of the revelations that have come from the development of co-dfns. These revelations came in the form of some quite high level development concepts for us to chew on and there are sure to be some interesting conversations as a result.

Aaron Hsu pontificates

The future is here, the future is now and the future is cross-platform. Richard Smith brought us yet another tool in the future tool box for Dyalog: Cross-Platform Configuration Files. The project is in early development and so the majority of the talk became an interesting debate into the pros and cons of various ideas for the format. XML, JSON, YAML or another – who will win? Only time will tell…

Geoff Streeter regales and informs about shared code files

Geoff told us the story of how a request to have functions loaded on demand led him to the germ of the idea and eventual implementation of shared code files. The audience was attentively, silently listening and the air of the Damgårdsalen was that of a village gathered around listening to their elder.

Richard Smith returned to show us some datetime functions to help us find out whether or not it is yet Christmas (SPOILER: It’s not Christmas yet). His slick demonstration reassured us that handling dates and times in Dyalog can and will be as painless as the idiosynchrosies of time and calendars permit – again after some details have been worked out.

Now we are adjurning for dinner. Later this evening we will be puzzling some puzzles in the APL Team Contest, hosted by members of Liceo Scientifico GB Grassi Saronno (a scientific high school in Italy). Don’t forget to check out tomorrow’s blog post to see how things went!

Dyalog ’19: Sunday 8 September

Welcome back

This year once again the Dyalog User Meeting returns to beautiful Elsinore in Denmark. The historic seaside city is home to Kronborg castle, famously immortalised in Shakespeare’s Hamlet – and Kromberg castle, where Morten lives. We are holding the user meeting at Konventum in the western outskirts of Elsinore. It features winding corridors adorned with contemporary Danish art and many comfortable seating areas conducive to social engagement, so we hope that delegates will find themselves meeting new people and conjuring beautiful new ideas as the week progresses.

Social spaces are everywhere at Konventum

Workshops

Today six half-day workshops were held, with topics ranging from source code management and graphical interfaces to cutting-edge APL techniques which have become available in the last decade of APL extensions in Dyalog.

Morten and Adam’s morning workshop focused on helping users collaborate on code with text-based source files. Adám introduced the ]LINK user command and Morten showed the way with Git – both guided the adoption of these technologies with worked examples.

We saw the delegates shine with their understanding of APL in the workshop on grouping and processing text. Nic helped us to understand the differences between partition and partitioned enclose. Powerful search and replace with ⎕R and ⎕S was elucidated by Richard Smith and, in this section, the ability of the participants to ask exactly the expected questions made the progression to understanding relatively smooth.

Brian Becker of the tools group and new recruit Josh David teamed up to introduce users to the new HTMLRenderer, which allows APLers to use web technologies to create cross-platform graphical user interfaces in Dyalog.

Adám points us in the right direction with function trains

Function trains are a relatively recent addition to the APL syntax, and with their terseness people can find them daunting both to read and write. However, once again we saw delegates stepping up to the challenge and finding creative ways to solve problems. It was a joy to see the creativity on display and the variety of approaches people took to solving the problems using only function trains. Marshall gave some details on the use of the rank operator , and despite this formidable challenge of understanding, by the end people were starting to grasp the power of this operator.

The morning’s HTMLRenderer workshop was mirrored by another GUI workshop in the afternoon. Michael and Chris Hughes worked to help people take their ⎕WC graphical interfaces, for Microsoft Windows, and get them to work on MacOS and Linux by using their qWC functions.

The mainframe is now the cloud, and with the ability to share great computing resources has come the need to learn another sizeable set of technologies. Morten and Norbert Jurkiewicz helped to clear some of the fog on this recent computing paradigm.

Things to come

For the rest of this week there will be many presentations from Dyalog employees and users – as well as an APL Team Challenge, the Viking Challenge, and of course the Banquet dinner on Wednesday evening.

We will be continuing to publish short daily recapitulations to give you a flavour of the talks and events of each day. However, if you are too impatient even for that, we will be streaming Monday morning’s talks live from 09:00 to 10:45 (07:00 to 08:45 UTC). Also, on Wednesday morning between 11:00 and 12:00 (09:00 to 10:00 UTC), the 2019 APL Problem Solving Competition will be concluded as this year’s grand prize winner Jamin Wu will be presented with his prize and will talk about his experience with the competition. These streams will be available to watch live on dyalog.tv, so make sure to tune in if you don’t want to wait until the talks are published later this year.

Dyalog ’18 Videos, Final Week

Welcome to the eighth – and final – week of recordings from the Dyalog User Meeting in Belfast! I’d like to take this opportunity to thank all the speakers who helped make Dyalog ’18 another valuable experience! Fittingly, we’re wrapping up with a larger, and even more varied collection than usual: two talks by members of the Dyalog team, two by users of Dyalog APL, and one from the British APL Association – five in total!

1. There are big changes afoot at the British APL Association and Paul Grosvenor, the chairman of the BAA, took to the stage in Belfast to tell us about them. Most importantly, the Vector magazine is going online after 25 years as a printed publication – and the removal of printing costs means that membership is now free! An annual conference in April/May is being planned. There has never been a better time to join the BAA, and you don’t need to live in the UK to do so! In addition to watching Paul’s talk, you can read about many of the changes here and on the new vector website (coming soon!).

2. Arianna Francia from SimCorp Italiana works in one of the largest APL development teams in the world. IFRS 9 is an International Financial Reporting Standard (IFRS) promulgated by the International Accounting Standards Board (IASB). Fortunately, although Arianna’s talk is titled “The IFRS 9 Project”, she completely avoids the subject of IFRS, and instead offers valuable insights into how a rapidly growing team, faced with an extraordinarily complex project, adapted and adopted a combination of agile practices and ideas from Ed Catmull’s book Creativity, Inc, which is about his experience managing Pixar.

3. For many Dyalog users, the least attractive aspect of Dyalog’s MiServer as a tool for cross-platform user interfaces is that it comes with a completely new set of controls or “widgets”, which essentially means you will need to rewrite your ⎕WC-based user interface. If you are facing this problem, Chris and Michael Hughes have very good news for you: in their talk “⎕WC on the Web”, they demonstrate a new tool that emulates ⎕WC, allowing you to create your UI in a web browser or the HTMLRenderer component included with recent versions of Dyalog APL.

4. Nested arrays make it easy – sometimes, too easy – to represent tables as 2-dimensional arrays. However, if each column of a matrix has the same data type, there are very significant savings to be had in both space and time if you “invert” such tables and represent them as a list of vectors, each containing the values for one column. In his talk on “Inverted Tables”, Roger Hui evolves a set of short, elegant and efficient functions for common operations on inverted tables.

5. It appears that the APL community came very close to losing John Scholes after he read the September 1989 edition of British Computer Journal special edition on Lazy Functional Languages and was struck by the beauty of functional programming. Fortunately, John decided to work on functional extensions to APL, and came up with dfns. This new notation was added to APL in 1996, only six years after Haskell 1.0 appeared. In his talk entitled “dfns, past present and future”, John revisits the early days and muses about things that could have been done differently, but quickly moves on to talk about ideas for future extensions to dfns, like guarded guards, where clauses, and optional type specifications.

I hope that you enjoy watching your choice of recordings as I have enjoyed revisiting them in order to write about them. As you may have noticed, we have taken a break from webinars while we have been rolling the Dyalog’18 recordings out. Now that we’re done, remember to set time aside at 16:00 (U.K. time) on the 3rd Thursday of each Month to follow the webinar series. The next webinar will be on Thursday February 21st: a presentation by our CTO Jay Foad, on his adventures as a participant in the Advent of Code programming competition, which was held in December.

Summary of this week’s videos:

New York Dyalog Meetup

I am very pleased to announce the creation of the New York Dyalog APL Meetup group, details of which can be found online at https://www.meetup.com/New-York-Dyalog-APL-Meetup/. The meetup has been created and is organised by Paul Mansour, who is also sponsoring the venue for the inaugural meetup, scheduled for 6-9pm on Thursday February 7th, at Alley, 119 West 24th Street, New York. If you are interested in meeting APL users in the New York area, please join the Meetup group so that you will be notified of future events. Please sign up for events that you intend to attend so we know you are coming!

Meetup is a service used to organize online groups that host in-person events for people with similar interests, including programming languages. In addition to the New York group, there is also an APL Meetup group in Frankfurt which meets regularly. We welcome the creation of more local meetups! If you create one in your area, remember to inform us at Dyalog so that we can add a link from our event calendar, and arrange to stop by and speak when we are in your neighborhood!

The program for the meetup on February 7th is as follows:

6:00-6:30pm: Time for Networking

6:30-8:00: Morten Kromberg: New Ways of Working with APL

When you are busy solving problems, new technology can be an unwelcome distraction – but every now and again technologies appear which have the potential to make development, maintenance or distribution significantly easier. Morten will demonstrate some of the new ways of working with APL that have become available in the last few years, and also discuss likely features in the next couple of releases of Dyalog APL: 17.1 in Q2 of 2019 and 18.0 in 2020.

8:00-8:15 Short Break

8:15-9:00 Paul Mansour: Keeping it Simple – A Git Workflow for APLers.

Abstract: Git is great, but the newcomer can easily drown in a sea of commands and options. Git doesn’t tell you when or why to branch, when or why to merge or rebase, how to version your project or prepare a release. AcreFlow is a radically simplified Git workflow that answers these questions. It is implemented in Dyalog APL so you can branch, commit, and put out new versions directly from the APL session.

 

Dyalog ’18 Videos, Week 7

Our stated goal has been to provide variety each week, and I’m happy to say that I have not been able to find any kind of theme in this penultimate set of presentations from Dyalog’18 in Belfast. Although the three technologies presented are all very useful, that is the only thing that they seem to have in common!

FlipDB is a relational database management system that was designed to solve complex data problems from the mortgage and asset finance business, but has grown into a general-purpose toolkit for working with tables. As Paul Mansour writes on the FlipDB web page, mortgages push the limits of many systems due to the quantity of data items, variety of products, constant innovation, and a never-ending stream of file formats and standards. Answering his own question, “What is wrong with SQL?”, Paul shows how easy it is to solve a number of programming puzzles from the Alteryx Weekly Challenge web site using the combination of array and object-orientation available in FlipDB. Just think about how difficult it would have been to do the same with SQL!

Co-dfns is a PhD project at Indiana University, executed by Aaron Hsu and partly funded by Dyalog Ltd. The aim is to build a concurrent compiler for dfns. Aaron regularly presents progress reports at Dyalog user meetings. This year, his focus has shifted to wrapping up demonstrations of completeness and other reporting required to finish his thesis – this will describe his compiler, which is designed to self-host on Data Parallel hardware. Who could have thought that analysing a lexically-scoped, dynamic, language could be so interesting (at least if you want the compiled code to run fast)?

A Jupyter Notebook is a formatted document containing live code, equations, visualizations and narrative text. As Adám Brudzewsky explains in his talk, Jupyter is an open framework that supports “language kernels” for many languages, including Dyalog APL. In other words, a notebook can contain APL expressions and their results – and the expressions can be edited and re-executed by the reader on demand. The expressions can either return textual output which is incorporated into the document, or HTML including SVG graphics – for example, output from SharpPlot or any tool that can produce HTML output.

Summary of this week’s videos: