Selective Expression Tracing

Operator trace from dfns.dws is a simple and low-tech tool for displaying intermediate results of functions, during the evaluation of an expression. For example, what’s going on here?

      (+⌿ ÷ ≢) 1 2 3 4

We can guess that the above fork is computing the average of the right argument. To see how this works:

      )copy dfns trace

⍝ Rename to reduce clutter

      t ← trace

⍝ Bind each "tine" of the fork with t to see what's happening

      (+⌿t ÷t ≢t) 1 2 3 4
≢  1 2 3 4  =>  4
+⌿  1 2 3 4  =>  10
10  ÷  4  =>  2.5

⍝ But how is that +⌿ reduction evaluated?

      (+t⌿ ÷ ≢) 1 2 3 4
3  +  4  =>  7
2  +  7  =>  9
1  +  9  =>  10

As a second example, what does the atop (~⍷) do in the following function to remove superfluous '.' characters from its argument?

      {('··'(~⍷)⍵)/⍵} 'squeeze·····me'

      {('··'(~⍷)t⍵)/⍵} 'squeeze·····me'
··  ~⍷  squeeze·····me  =>  1 1 1 1 1 1 1 0 0 0 0 1 1 1

…and so forth. Notice how t can be injected to the right of any of the functions in an expression.

For more examples, see the notes for trace.

Data-driven Conditionals

ACKNOWLEDGEMENT: My thanks to Andy Shiers for simplifying the examples

Visitors to APL from other languages are sometimes a bit sniffy about our use of numbers (1 0) for Boolean values True and False. They’re further bemused by our fondness for moving conditional processing out of code and into data. For example, instead of:

   0=2|⍵: ⍵÷2 ⋄ ⍵       ⍝ even number: half

we might say:

   ⍵ ÷ 1+0=2|⍵          ⍝ half if even

At first blush, this seems a bit loony; such transformations appear to produce code which is both slower and more obscure! Here’s another example:

   2|⍵: 1+3×⍵ ⋄ ⍵       ⍝ odd: one more than its triple


   p ← 2|⍵              ⍝ parity
   p + ⍵ × 1+2×p        ⍝ one more than triple, if odd

But think arrays! Data-driven processing of conditionals is inherently parallel and so performs significantly better for large arrays.

The Collatz conjecture asserts that the sequence “half if even, else one plus triple if odd” always converges to 1. For example, for the number 7 this sequence is:

   7 22 11 34 17 52 26 13 40 20 10 5 16 8 4 2 1

To test this up to a million we can borrow time, a coarse-grained timing operator, from dfns.dws:

          {                 ⍝ scalar function
              2|⍵:1+3×⍵     ⍝ odd              
              ⍵÷2           ⍝ even             
          }⍣{⍺=1}¨⍵         ⍝ applied under each

          {                 ⍝ vector function                                              
              p←2|⍵         ⍝ parity                   
              u←⍵÷2-p       ⍝ half of evens            
              v←p+u×1+2×p   ⍝ 1+3× odds                
              ∪v~1          ⍝ without 1s and duplicates
          }⍣{⍺≡⍬}⍵          ⍝ applied to whole vector                           

      scalar time ⍳1e6      ⍝ scalar processing 8¾ minutes
      vector time ⍳1e6      ⍝ array processing 8¾ seconds

In the above, time and are high order “operators”.

Defined operator time takes a function left operand, which it applies to its argument, returning the execution time in minutes and seconds.

Primitive operator (power or fixpoint) applies its left operand function repeatedly until its right operand function, which is supplied with the result () and argument of each application, returns True (1).

For further discussion of the Collatz function, see

Why Dyalog ’14 was a Shear Delight


Recent versions of Microsoft Windows support touch screens, which of course means that applications can respond to events originating from touches. Microsoft calls these events “gestures”.

Dyalog decided to add support for gestures to version 14.1 and so projects were planned, designs were designed, code was coded and, at Dyalog ’14 (#Dyalog14), a demonstration was demonstrated. The video of that demonstration is here

The three most interesting gestures are “pan”, “zoom” and “rotate”; I needed a good demo for the rotate gesture. I had an idea but was unsure how to implement it, so I decided to ask the team. Here’s the email I sent (any typos were included in the original email!):

“Does any have code (or an algorithm, but code ‘d be better) to rotate a matrix by an arbitrary angle. The resulting matrix may then be larger than the original but will be filled with an appropriate value.
Does the question makes sense? For the conference I want to demonstrate 14.1 support for Windows touch gestures, one of which is a “rotate” gesture, and I thought that a good demo would be to rotate a CD cover image when the gesture is detected. So I need to be able to rotate the RGB values that make up the image.”

There were a number of helpful responses, mainly about using rotation matrices, but what intrigued me was Jay Foad’s almost throw away comment: “It’s also possible to do a rotation with repeated shears”

That looked interesting, so I had a bit of a Google and found an article on rotation by shearing written by Tobin Fricke. The gist of this article is that shearing a matrix three times (first horizontally, then vertically and then again horizontally) effects a rotation. An interesting aspect of this (as compared to the rotation method) is that no information is lost. Every single pixel is moved and there is no loss of quality of the image.

I’m no mathematician, but I could read my way though enough of that to realise that it would be trivial in APL. The only “real” code needed is to figure how much to shear each line in the matrix.

Converting Fricke’s α=-tan(Θ/2) and ϐ=sin(Θ), we get (using for Θ):


and so, for a given matrix, the amounts to shift are given by:


where is either a or b, depending on which shift we are doing.

Wrapping this in an operator (where ⍺⍺ is either or ):

(⌊⍺×(⍳≢⍵)-0.5×≢⍵)⍺⍺ ⍵

Our three shears of matrix , of size s, can therefore be written as:

a⌽shear b⊖shear a⌽shear(3×s)↑(-2×s)↑⍵

Note the resize and pad of the matrix before we start so that we don’t replicate the edges of the bitmap. The padding elements are zeros and display as black pixels. They have been removed from the images below for clarity.

The final rotate function is:

shear←{(⌊⍺×(⍳≢⍵)-0.5×≢⍵)⍺⍺ ⍵}
a←-3○⍺÷2 ⋄ b←1○⍺ ⋄ s←⍴⍵
a⌽shear b⊖shear a⌽shear(3×s)↑(-2×s)↑⍵

Let’s look at a 45° rotate of a bitmap called “jd”.

jd.CBits←(○.25)rotate jd.CBits

Prior to performing any shears our image is:


After the first shear it becomes:


After the second shear it is:


And after the third shear it is:


All nicely rotated.

Thursday 25 September at Dyalog ’14

This was originally posted to Catalyst PR’s Facebook page and is reproduced here to make it accessible to people who don’t have Facebook accounts.

For information on the presentations at Dyalog ’14, see

On Thursday 25 September we were in for an impressive presentation called School Laser System with NFC Registration and Alerts by Chris “Ziggi” Paul, Laser Learning Ltd (U.K.).

Ziggi has previously developed other applications in Dyalog APL – most notably an ePortfolio-based nursery training solution for the Childcare company (see Dyalog’s case study at which won the Nursery Supplier/Innovator 2008 Award at the Nursery Management Today Awards.

Since then the company has grown substantially and have branched out into Carer’s Careers – a company that is focused on training people who work with caring for others – and now the School Laser System with Near Field Communications (NFC).

The background for the School Laser System with NFC is that the principals are now also working with a new Academy free school. The School has 728 pupils, aged 4-11, and they need to be accounted for at all times during the day. The school is located over two main sites and has an external catering hall and a swimming pool. It also provides before-school and after-school clubs and mini-bus services.

The school “LASER” system is designed as a complete communication and information system between parents and school staff; recently it has been expanded to NFC in order to provide teachers and staff with up-to-date, relevant information. Each pupil in the school is issued with an NFC wristband and tags for their bags and key items of the uniform. Staff use NFC-enabled mobile phones or NFC readers connected to PCs to log a pupil at key points, for example, on and off a bus or at the dining hall. The NFC tags let the lunch staff know what food order the parents have requested each day and can give a full itinerary of clubs and school activities. Alerts of allergies or other critical medical information is displayed when a tag is scanned.

Ziggi showed us the parents’ interface to the system, as well as that used by the staff. The security around the system is obviously very high, using encryption and secure log-in. There is no data stored on the wristbands, bag tags or uniform items. If a child has forgotten to bring their wristband, a substitute – which only works for that day – is issued.

I believe that the system has great potential. There is a market not only for other UK infant and junior schools to implement this applications, but catering firms supplying school dinners could also benefit greatly from using this application.

NOTE for readers not familiar with the requirements for UK school staff concerning reporting, tracking, tracing, accounting for, etc. all children at all times – and who may have concerns regarding “big brother” tracking of children’s movements during the time they’re at school or attending after school clubs, etc.: This system is developed for tracking children aged 4-11. The application saves the school staff an enormous amount of admin. Parents love it, and as for issues around data protection, for example, medical information, no data is stored on the items the children carry around (wristband, bag tag or uniform items). It is already common practice that children with allergies and other medical conditions have their own little poster with their picture, age, condition and actions required in case of an incident posted in the classroom and the teacher room.

This is my final post from Dyalog ’14 – see you at Dyalog ’15, which seems to be going in the direction of Italy (TBC).

Postcard from Dyalog ’14 – Thursday

This is the last day of Dyalog ’14 so this is the last of six daily postcards from Eastbourne – we hope you have enjoyed getting a flavour of things as they happened. We expect there will be more discussion to come once we’re back and rested!

Last Night’s Banquet

Yesterday culminated in the traditional banquet dinner – the highlight of the social element of the user meeting. This year it was held at The Grand Hotel Eastbourne and consisted of a Champagne reception and formal three course dinner with live music from a string trio.

During the banquet the winners of the scavenger hunt were presented with their prizes – the ducks and other “British” items we saw on Sunday’s postcard! There was also a quiz to identify Dyalog employees from a list of clues. Exquisite food and wonderful company meant an exceptionally late night (early morning!) …

Delegates arrive at The Grand Hotel

Delegates arrive at The Grand Hotel

Banquet Dinner

Banquet Dinner

Team Herring celebrate their win in the quiz

Team Herring celebrate their win in the quiz

Moris and Luana with their prizes

Moris and Luana with their prizes

Discussion Point: Read-only Component Files

In his presentation today on component files and the Dyalog File Server (DFS), Richard mentioned a perhaps little-known problem with restricting access permissions on component files for users that nominally only ever read them. Setting read-only permission at the file level – that is, imposed by the operating system – could add a level of security to the file but it is in fact the case that component file reads can occasionally require write access to the file.

For example, suppose User A is reading and writing to a journaled component file and User B is simply ⎕FREADing it. If User A is interrupted whilst updating the file (perhaps the APL session is killed or a network connection is lost) and User B is the next to read it then as part of that read the interpreter will detect the journal and complete the unfinished update – a process which requires write access. If the systems administrator who set up the file permissions for User B had only granted read-only file permission to the file then the ⎕FREAD would fail.

Richard explained that this restriction might be lifted in future and that read operations would be truly read only – even with a journal present – but there is another issue to be aware of too. Access to component files is controlled using file locks. As might be expected, exclusive access is obtained on an exclusive tie and shared access is obtained on a share tie. In addition, writes to a share tied file require exclusive access for the duration of the write so that no other client is able to read it while it is being updated. Exclusive locks are write locks and thus require write permission. Therefore, if there is no write access to the file the interpreter cannot exclusively tie it – it can only share tie it.

Where file access has to be locked down as securely as possible, the DFS is able to resolve the above issues and more besides – client access to all component files is only possible via APL, users are fully authenticated and individual rights are controlled by the access matrix.

And Also …

Some of the other things we saw and heard today:

  • Andy showed us how to close trace windows without cutting back the stack
  • Ziggi pointed out that a loss rate of one in 700 is acceptable when delivering wine but not when delivering children
  • We would like your suggestions and comments about the user meeting programme and structure, and preferred location for next year – please email us at


Eastbourne has not let us down and even the British weather (which is always unpredictable) defied the odds and rendered the ponchos and umbrellas unnecessary. The View Hotel started life as a conference centre and was purpose built for holding meetings such as ours. We have certainly found it to be an ideal venue and feedback from delegates has been overwhelmingly positive.

The View Hotel

The View Hotel

The View Hotel

The View Hotel

The picture of the pier below was carefully framed not to show the fire damaged part! The fire that destroyed part of it back in July was headline news; the pier is closed as a result but should partially reopen next week. Perhaps we will have to return when it is fully restored!

Eastbourne Pier

Eastbourne Pier

Bandstand with The View Hotel behind

Bandstand with The View Hotel behind

Signing Off

And so Dyalog ’14 has come to an end. It has been the best-attended Dyalog conference anyone can remember and reflects the growing success of Dyalog and its users. Here begins the countdown to Dyalog ’15!

Wednesday 24 September at Dyalog ’14

This was originally posted to Catalyst PR’s Facebook page and is reproduced here to make it accessible to people who don’t have Facebook accounts.

For information on the presentations at Dyalog ’14, see

During Dyalog ’13, Dyalog Ltd announced an agreement with Syncfusion Inc. The agreement stated that Syncfusion’s Windows Presentation Foundation (WPF) and JavaScript libraries would be bundled with Dyalog version 14.0 (see

On Wednesday the 25th it was, therefore, a pleasure to attend the presentation by Claus Madsen of Fine Analytics (Denmark), where he presented the company’s InRisk Asset Allocation Platform.

The application is written purely in Dyalog and is aimed at professional investors. Fine Analytics have taken advantage of the WPF capabilities and Syncfusion controls available with Dyalog 14.0 to build a generic asset allocation platform with special focus on the professional investor. A specially designed .NET class facilitates the separation of the administrative and analytical sections of the application and a flexible calculation engine that allows a user to define type/structure/order is an integral part of the platform. All assumptions can be controlled from the professional-looking user interface; the user interface can also be used to change/import/construct/delete/add/copy/export a portfolio as well as add new assets in asset groups.