Extending Structural Functions to Scalars

Posted on by

Traditionally, the set of monadic reversing or reflecting primitives, Reverse-First (⊖), Transpose (⍉), and Reverse (⌽) apply to entire arrays and are defined as identity functions on scalar arguments. Dyalog v19.0 extends the definitions to provide equivalent reflections on scalars.

Character Data

We expect that the new transformations will be most useful on characters. For example:

      (⍉'A')(⌽'P')(⊖'L')
ᗉꟼΓ

Note that you can apply the same transformation to all the items of an array using Each (¨) or Rank (⍤0):

      ⌽¨ 'New APL?'
Ͷɘw Aꟼ⅃⸮
      (⊖⍤0) 'CAN HAPPEN SOON!'
C∀N H∀ЬЬEN ꙄOOͶ¡

Composition allows the combination of reflections to perform rotation:

      ⊖∘⍉¨ '→A/common'  ⍝ 90° clockwise rotation
↓ᗆ\ᴒoᴟᴟoᴝ
      ⌽∘⊖¨ '→A/common'  ⍝ 180° rotation
←∀/ɔoɯɯou

We can combine the above techniques with additional structural functions, including reflection of the entire array, to achieve more advanced effects that are often needed in modern user interfaces:

      ⌽¨ ⌽ 'New APL?'     ⍝ mirror
⸮⅃ꟼA wɘͶ
      ⊖∘⍉¨ ⍪ '→A/common'  ⍝ vertical
↓
ᗆ
\
ᴒ
o
ᴟ
ᴟ
o
ᴝ
      ⌽∘⊖¨ ⌽ '→A/common'  ⍝ upside-down
uoɯɯoɔ/∀←

Numeric Data

Although the transformations are more easily applicable to characters, many numbers are also in the domain of the extended functions:

      ⌽¨ 1.618 2.71828 3.1415
816.1 82817.2 5141.3
      ⌽¨ 3J4 0J1 0.5
4J3 1 5
      ⌽∘⊖¨ 60 69 908    ⍝ 180° rotation
9 69 806
      ⍉⌽ 8              ⍝ 90° counter-clockwise rotation
1.797693135E308

Notes

Character Data

  • Although the new definitions are available in both 32-bit and 64-bit Unicode editions of Dyalog, very few characters can be reflected in the Classic edition.
  • A TRANSLATION ERROR will be signalled if the required result cannot be represented.

For example, using the Classic Edition where the Rank operator is represented as ⎕U2364:

      (⊖ ⎕U2364 0) 'PHI!'
bHI¡
      (⊖ ⎕U2364 0) 'ABC'
TRANSLATION ERROR: Unicode character ⎕UCS 8704 not in ⎕AVU
      (⊖⎕U2364 0)'ABC'
      ∧

Numeric Data

  • The result of numeric reflections can depend on the value of ⎕FR.
  • A DOMAIN ERROR will be signalled if the required result cannot be represented.

For example, beginning with the default value ⎕FR←645:

      ⌽ 1.2345E67
DOMAIN ERROR
      ⌽1.2345E67
      ∧
      ⎕FR←1287
      ⌽ 1.2345E67    ⍝ 76×10*5432.1
9.56783313E5433

Conclusion

Although it is extremely unlikely that real applications rely on the current behaviour, the extensions are potentially breaking changes and are, therefore, being released as part of a major version upgrade.

We are somewhat surprised that these obviously useful extensions have been ignored by the APL community for such a long time, and are very pleased to finally make them available to commercial, educational and hobbyist users. Please contact ʇɹoddns@dyalog.com if you would like to test a pre-release of Dyalog v19.0 and help us understand the potential impact on existing applications.

Email, RSS Follow

APL Seeds ’23: Wednesday 22 March

Posted on by

Pie chart of APL Seeds '23 registrant prior knowledgeLast Wednesday we hosted APL Seeds ’23, the third of our free online user meetings aimed at new and prospective users of APL. In attendance was a mix of new users and the APL-curious, as well as more experienced APLers answering questions and imparting wisdom in the chat and during the post-event meetup.

Rich Park began the meeting with an introduction to the basic syntax of the language and a sample solution to a problem from last year’s APL Problem Solving Competition – from data about the schedule of an online event, and the times people joined the event, which sessions did each user attend? He demonstrated how the thought process of a solution could map neatly onto an executable APL expression. In particular, he showed how an elegant solution tends towards using primitives directly with array argument values, as opposed to abstracting the solution for one piece and explicitly looping over the collection.

Adám Brudzewsky then explored how to remember unfamiliar ideas. He presented some mnemonic devices linking the glyphs that make up the core APL language with both the concepts that they represent and their locations on the APL keyboard. Whether they are two related concepts represented by the same symbol, or if the dyadic form is the same as monadic with a default left argument, or if two unrelated functions use the same glyph, these tricks are really useful for memorising the fundamental building blocks of APL.

Screenshot of Adám Brudzewsky's presentation

A still screenshot doesn’t do justice to the wonderful animations in Adám’s presentation.

We then heard about learning APL from the perspective of a non-programmer. Richard Savenije is a project director for infrastructure projects in a construction company. After a stint of programming in C at IBM, Richard decided he did not want to pursue programming any further. When he left, he took an APL manual with him, as he had been intrigued by the glyphs. It sat untouched for 28 years, until he recently decided to start learning to solve problems with APL. He told us about the new perspectives he has gained while using APL, and parallels with the type of problem solving he does in his day-to-day life.

APL media often shows the symbols, interesting approaches to algorithms, and other ways in which APL is fascinatingly unique in the world of programming languages. However, this often leaves people wondering whether they could use this in the more pedestrian tasks of automation, data processing, and application development. The final two talks of the day addressed the more practical use of APL.
You might have heard Stefan Kruger’s name before – he wrote the excellent Learning APL book and gave a presentation at APL Seeds ’22. He acknowledged that the domain of APL is largely seen as “super tight” numerical algorithms and array transformations. This year he explained the convenience of HttpCommand and ⎕JSON for ingesting data from external sources, and explored the creation of user commands to provide a convenient interface for performing repetitive tasks. Stefan used these to rewrite a substantial collection of (mostly Python) utility scripts as APL functions and user commands, ultimately shrinking the code by a factor of 10!

Screenshot of Stefan Kruger's presentation

Stefan shows us how to construct user commands to conveniently perform repetitive tasks.

In the final presentation of the day, Josh David answered the question of “what does it look like when my APL Seeds come to fruition?” He began with a demonstration of some analysis of water potability data that turned out to look dangerous – or fake. Throughout his demonstration, we got a feeling for how to rapidly inspect and transform data to gain insights. He included plenty of very useful tips for new APLers, such as how to stop the stream of text into the session when you execute an expression that produces long output (Action > Interrupt in the Session menu…there are more tips like this on our website). Finally, he showed some full-stack graphical applications – his SQL-Browser and The Carlisle Group’s FlipDB – with examples of database connectivity, querying and analysis.

After the talks, there was an informal online meetup. This included lively discussion about many topics, including language design and terminology. We also discussed questions about teaching tacit programming, and heard perspectives from some APL beginners – if someone is learning the concept of functions for the first time, then composition of functions is probably not appropriate at that point.

We hope we were able to spark interest, give some clarity to people’s image of APL, and inspire you to start or continue learning and using APL. Relevant materials have been uploaded to the APL Seeds ’23 webpage, including links to recordings of the presentations on dyalog.tv.

Email, RSS Follow

Structural vs. Mathematical “Under”

Posted on by

The APL Farm has recently hosted discussions about the proposed under operator, asking “why Dyalog hasn’t implemented it a long time ago”. I claimed that an important reason was that Roger Hui wasn’t a big fan of the proposed extension known as “structural under”. Stefan Kruger then asked me why that was so. Since Roger isn’t able to respond himself, I’ll do my best to explain my own understanding of the issues.

For anyone not familiar with the concept of under, or dual as it was known when first implemented in SHARP APL, it is an operator that provides a way to express a common mathematical pattern, where a computation or proof is simplified by first translating the input domain, applying a function, and then performing the inverse transformation. For example, if we use the symbol ⍢ to denote under, have the translation function as the right operand, and the function that applies to the translated domain as the left, then we could use +⍢⍟ to demonstrate that multiplication is equivalent to addition under logarithm, which is why slide rules work:

      1 2 3 4 (× ≡ +⍢⍟) 0.1 1.1 17 100
1

Another example of simplifying multiplication using dual is the use of Fast Fourier Transforms to multiply very large numbers, as explained in the notes about xtimes in the dfns workspace.

It is easy to find examples of dualities outside the mathematical domain, such as midnight snacking: (open refrigerator door, remove food, close door) or Roger’s favourite example of medical surgery: apply anaesthetic to greatly simplify the operation, perform surgery and (very important) back out the anaesthetic.

I believe that Roger was of the opinion that under/dual should be restricted to functions that have well-defined inverses using the “mathematical” definition used in SHARP APL and the J language. We can model this with a defined operator, let us call it Um:

   Um←{⍺←{⍵ ⋄ ⍺⍺} ⋄ ⍵⍵⍣¯1⊢(⍵⍵ ⍺)⍺⍺(⍵⍵ ⍵)}

The extension known as structural under allows the transformation function to be a structural function, even though such functions do not have inverses in the mathematical sense. In this case, the left operand is applied to the subset of items selected from the right argument by the structural function. This is easy to model using a feature of some APLs known as selective specification, in which you can have a function to the left of an assignment whereby the assignment only modifies items selected from an array using so-called “structural” functions – those functions that select or reorder items of an array without doing math, such as ↑ ↓ ⍉ ⌽ ∊.

Let us call this Us:

   Us←{⍺←{⍵ ⋄ ⍺⍺} ⋄ w←⍵ ⋄ ((⍵⍵)w)←(⍵⍵ ⍺)⍺⍺(⍵⍵ ⍵) ⋄ w}

This is very useful, for example, we could negate all items of an array except the first two by writing:

      - Us (2∘↓) ⍳5
1 2 ¯3 ¯4 ¯5

The big question is whether Us and Um can be combined into the same operator, or two separate operators are needed. One problem is that ↓ already has a defined inverse in Dyalog APL, which pads with zeros, so Um returns something different:

      - Um (2∘↓) ⍳5
0 0 ¯3 ¯4 ¯5

If we want to combine the two into one, then we need a rule that allows us to decide which definition to use. Marshall Lochbaum‘s BQN language has a table of structural primitive functions; it performs a structural under when possible (according to an extensively described rule), and otherwise attempts a mathematical under.

Personally, this sort of conditional logic in the definition of a key primitive makes me very uneasy, and I think Roger was struggling with this as well. Unfortunately he passed away before we were able to finish argdiscussing the design. Another reason for feeling that the two operators are different is that, while it often makes sense for the transformation to apply to both arguments in the mathematical case, it is extremely rare for structural under. You cannot do:

       1 + Us (2∘↓) ⍳5

because when you apply (2∘↓) to the left argument, you get an empty vector. Instead, you need to turn the left operand into a monadic function to add one to the three trailing elements:

       1∘+ Us (2∘↓) ⍳5
1 2 4 5 6

This increases my personal unease about combining the two into a single operator. Even though this would work “just fine” most of the time, it would still make it difficult to explain exactly what is going on.

It is my opinion that APL needs to remain a rationalised mathematical notation if it is to remain useful and relevant in the long term. Deciding whether to implement a strict mathematical operator, a combined operator, or two separate operators (or do nothing for a while longer), is one of the key design decisions that we will be faced with in the near future.

Email, RSS Follow

Welcome Peter Mikkelsen

Posted on by

Peter joined Dyalog Ltd in mid-July 2022, shortly after finishing his bachelor’s degree in Computer Science from Aalborg University in Denmark. He didn’t originally plan to look for a job so soon, but when he saw that Dyalog Ltd had an opening involving three of his favorite things (C programming, interpreters, and APL) it was too good an opportunity to let pass and he was offered the job after a couple of online calls and interviews. Peter and his girlfriend had planned to move to Esbjeg in the south-west of Denmark for a while, so he thought it was great that Dyalog Ltd made it possible to work from home. He joined just in time to be able to participate in Dyalog ’22, which meant that he met lots of customers, which he considered a great experience.

His interest in APL started while he was at university – at one point he thought to himself: “This language looks so strange and different that it might be worth learning.” After spending some time playing around with it and participating in the 2021 APL Problem Solving Competition, he started writing his own little APL interpreter, which he says must have been the point of no return for him. Even in the first few months at Dyalog Ltd, he has learned a great deal about the language and its quirks, and a lot about implementation techniques.

Peter hopes to be able to help develop the language in the years to come, and to make sure dfns get the love they deserve, as he thinks that functional programming is definitely the way to go!

Even though he is part of the “young generation” of APLers, writing long trains and doing code golfing in APL has yet to catch his interest.

When not working, Peter is likely to be found taking a long walk, or trying out something computer science related that has caught his interest. Often, the code he writes in his free time is for the Plan 9 operating system, which in his mind is so elegant and simple that it is almost a shame it has so few users.

Email, RSS Follow

Welcome Silas Poulson

Posted on by

As is often the case these days, Silas found APL through online programming language discussions – both in various forums and through videos (Aaron Hsu’s Co-dfns and John Scholes’ Game of Life demonstatration in particular) – whilst attempting to understand the more esoteric ideas and their theoretical underpinnings provided by Scheme and other functional languages. After discovering APL, Silas attempted to use it for his Data Analysis coursework, sticking primarily to a small set of primatives. However, hitting WSFULLs with the datasets required meant that APL remained just a fun system to play with, which led to an understanding of where (⍸) in terms of other primatives (⍳ and ⍴).

Silas saw one of Dyalog Ltd’s adverts for a C programmer; this was a language he had learnt during his exploration of the Raspberry Pi and its associated add-on boards, so he applied. He joined Dyalog Ltd in June 2022 after graduating from Royal Holloway in England with a bachelor’s degree in Computer Science, and eased his way into exploring the internal workings of the interpreter, where he now spends most of his time.

When not programmaing, Silas can be found enjoying a good book and the occasional summer hike with friends and family.

Email, RSS Follow

Dyalog ’22 Day 5: Looking Forward

Posted on by

Links to recordings from this day of the user meeting are at the bottom of this post.

To begin the final day of the user meeting, Aaron Hsu presented some strategies and ways of thinking about processing data that come from text. While his specific focus is parsing APL code for the co-dfns compiler, he believes that a parallel data-flow approach will be useful across domains, for example, when processing JSON data, which is common in modern web communication.

Aaron talks about parsing data from text

Aaron talks about parsing data from text

Adám Brudzewsky, despite not being in the room, captivated us with a supremely convincing argument for why three primitives could be considered to be filling in “core language” gaps: the “Select” function (⍺⊇⍵) for elegant indexing; the “Depth” operator (F⍥k) as analogous to rank (F⍤k) for nested arrays; and the “Behind” (F⍛G) operator complementing other function compositions “Beside” (F∘G), “Atop” (F⍤G) and “Over” (F⍥G) to make tacit programming an even better extension of APL expressivity.

Adám proposes a new "behind" function composition operator

Adám proposes a new “behind” function composition operator

It was exciting to see a return for APL to its birthplace in academia. Jesús Galán López of the Technical Univerity of Delft told us his plan to use APL in metallurgical models and analysis of experimental data. He hopes to use Dyalog APL as a central connecting piece in education, science, engineering and research. This is one of the first projects to eventually be supported by the APL fund mentioned by Stine on Monday.

Jesús with a typical data analysis example in materials science

Jesús with a typical data analysis example in materials science

A living legend in APL education, Raymond P. Polivka presented his latest book aimed at high school students with no programming knowledge whatsoever. The clearly structured modules and exercises will be an invaluable resource for teachers and recruiters alike, and they have been developed with both a lifetime of experience teaching APL as well as very recent classes with real high school students. He is asking for feedback on a draft available at aplclass.com/book.

Ray Polivka introduces his new book aimed at high school students

Ray Polivka introduces his new book aimed at high school students

For the last talk of the user meeting, Rich Park took us on a whirlwind tour of the media and training materials being developed at the moment. He also commented on some of the work that he believes is needed to tie these resources together and make them more discoverable, and what might be done to improve the experience for new users trying to find things out for themselves.

Rich Park talks about media, training and outreach at Dyalog

Rich Park talks about media, training and outreach at Dyalog

In the afternoon we ended the user meeting with some workshops. We got to know about performance considerations in the context of the Dyalog interpreter with Aaron Hsu and Karter Kooner. There was a chance to get hands-on experience using one of the asynchronous, concurrent programming methods in Dyalog: Futures and Isolates with Morten Kromberg and John Daintree. Josh David and Brian Becker taught us about sharing APL tools by creating and publishing packages using Tatin.

It has truly been an incredible experience returning to in-person user meetings. The sense of community felt in Olhão this year was far beyond what most people predicted, not just in light of separation over the last two years, but also in looking forward to an exciting future for APL and Dyalog.

Today’s presentations (links to recordings will be added as they become available):

Email, RSS Follow