Shuffle Faster Please!

Andy reported that in the shuffle QA some functions take a long time:

m9249   “4½ days so far”
rankop  21.5 hours
m12466  26.3 hours
points  7.2 hours

Background: Shuffle QA is an intensification of the normal QA. The suite of over 1800 QA functions is run under shuffle, whereby every getspace (memory allocation) is followed by every APL array in the workspace being shifted (“shuffled”) and by a simple workspace integrity check. The purpose is to uncover memory reads/writes which are rendered unsafe by events which in normal operations are rare.

m9249

m9249 tests +\[a], ×\[a], ⌈\[a], and ⌊\[a]. It ran in under 3 seconds in normal operations; for it to take more than 4½ days under shuffle, getspace is clearly the dominant resource and hereafter we focus on that.

m9249 used expressions like:

assert (+⍀x) ≡ {⍺+⍵}⍀x

Since {⍺+⍵} is not recognized by the scan operator and therefore not supported by special code, {⍺+⍵}⍀x is done by the general O(n*2) algorithm for scans, AND on each scalar the operand {⍺+⍵} is invoked and a getspace is done. The improved, faster QA does this:

F ← {1≥≢⍵:⍵ ⋄ x ⍪ (¯1↑⍵) ⍺⍺⍨ ¯1↑x←⍺⍺ ∇∇ ¯1↓⍵}
assert (+⍀x) ≡ +F x

With such a change, the faster m9249 runs in 22 minutes under shuffle instead of over 4½ days, without any reduction in coverage.

The number of getspace are as follows: +⍀x takes O(1). The second expression {⍺+⍵}⍀x does ×\1↓⍴x separate scans, and each one takes O((≢x)*2) getspace; the total is therefore O(((≢x)*2)××/1↓⍴x) ←→ O((≢x)××/⍴x). The third expression +F is linear in ≢x and is therefore O(≢x) in the number of getspace. In m9249 the largest x has size 33 5 7. The following table shows the orders of complexity and what they imply for this largest x:

+⍀x O(1) 1
{⍺+⍵}⍀x O((≢x)××/⍴x) 38115 = 33 × ×/ 33 5 7
+F x O(≢x) 33

The ratio between 38115 and 33 goes a long way towards explaining why the time to run m9249 under shuffle was over 4½ days and is now 22 minutes. (Why does m9249 still take 22 minutes? It tests for the four functions + × ⌈ ⌊, for various axes, for different datatypes, and for different axis lengths.)

rankop

Another shuffle sluggard rankop took 47 hours. rankop tests expressions of the form f⍤r⊢yand x f⍤r⊢y for various functions f. The techniques used for reducing the number of getspace differ from function to function. We look at x↑⍤r⊢y as an illustration.

assert (4↑⍤1⊢y) ≡ 4{⍺↑⍵}⍤1⊢y

⍴y was 2 7 1 8 2 8. The expression took 7.6e¯5 seconds normally and 7.5 seconds under shuffle. The left hand side requires O(1) getspace, the right hand side O(×/¯1↓⍴x). The expression was changed to:

assert (4↑⍤1⊢y) ≡ 2 7 1 8 2 4↑y

Feels like cheating, doesn’t it? :-) The new expression takes 0.115 seconds under shuffle.

m12466

m12466 tests n⌈/[a]x (and n⌊/[a]x). Previously, it did this by comparing n⌈/[a]x against n{⍺⌈⍵}/[a]x for x with shape 7 17 13 11, for each of the four axes, for two different values of n, and for the 1-, 2-, 4-, and 8-byte numeric datatypes. It required 5238426 getspace and 26.3 hours to run under shuffle.

F←{p⍉⍺⍺⌿(⊂(⎕io-⍨⍳|⍺)∘.+⍳(1-|⍺)+(⍴⍵)[⍵⍵])⌷(⍋p←⍒⍵⍵=⍳⍴⍴⍵)⍉⍵}

F is a dyadic operator. The left operand ⍺⍺ is or ; the right operand ⍵⍵ is the axis; and n(⌈ F a)x is the same as n⌈/[a]x. n(⌈ F a)x mainly does n⌈⌿x; other axes are handled by transposing the axis of interest to be the first, then doing n⌈⌿x, then transposing the axes back into the required order. n(⌈ F a)x is much more complicated than n{⍺⌈⍵}/[a]x but has the advantage of using only O(1) getspace.

The revamped m12466 function uses 13717 getspace and 2 minutes to run under shuffle.

points

points is a function to test monadic format () for different values of ⎕pp.

points←{⍺←17 ⋄ ⎕pp←⍺
  tt←{(⌽∨\⌽⍵≠⍺)/⍵}                   ⍝ trim trailing ⍺s.    
  ~0∊∘.{                             ⍝ all pairs (⍳⍵)       
    num←'0'tt↑,/⍕¨⍺'.'⍵              ⍝ eg '9.3'             
    num≡⍕⍎num                        ⍝ check round trip     
  }⍨⍳⍵
}

The expression is 14 15 16 17 points¨ 100, meaning that for each of the 100 numbers num as text,

  1.1   1.2   1.3   1.4   1.5     1.100
  2.1   2.2   2.3   2.4   2.5     2.100
  3.1   3.2   3.3   3.4   3.5 ... 3.100
...
 99.1  99.2  99.3  99.4  99.5    99.100
100.1 100.2 100.3 100.4 100.5   100.100

a “round-trip” num≡⍕⍎num is evaluated. The expression requires 1.3 million getspace and 7.2 hours to run under shuffle.

A much more efficient version with respect to getspace is possible. Let x be a vector. ⍕¨x requires O(≢x) getspace; ⍕x requires O(1) getspace and, like ⍕¨x, formats each individual element of x independently.

points1←{
  ⎕io←⎕ml←1                                                 
  tt←{(⌽∨\⌽⍵≠⍺)/⍵}                   ⍝ trim trailing ⍺s     
  x←1↓∊(' ',¨s,¨'.')∘.,'0'tt¨s←⍕¨⍳⍵  ⍝ numbers as text      
  y←⍎x                                                      
  {⎕pp←⍵ ⋄ x≡⍕y}¨⍺                   ⍝ check round trip     
}

14 15 16 17 points1 100 requires 11050 getspace and less than 2 minutes to run under shuffle.

What to do?

Andy says that the shuffle QA takes over 2000 hours (!). The shuffle sluggards will be tackled as done here by using more parsimonious APL expressions to compare against the aspect being QA-ed. Going forward, a new QA function will be run under shuffle as it is being developed. The developers will be unlikely to tolerate a running time of 4½ days.

Loops, Folds and Tuples

For-loops
Given an initial state defined by a number of variables, a for-loop iterates through its argument array modifying the state.

    A←... ⋄ B←... ⋄ C←...       ⍝ initial state
    :For item :In items         ⍝ iterating through array "items"
        A←A ... item            ⍝ new value for A depending on item
        C←C ... A ... item      ⍝ new value for C depending on A and item
        ...                     ⍝ state updated
    :EndFor
    A B C                       ⍝ final state

In the above example the state comprises just three variables. In general, it may be arbitrarily complex, as can the interactions between its various components within the body of the loop.

Dfns
Dfns don’t have for-loops. Instead, we can use reduction (or “fold”) with an accumulating vector “tuple” of values representing the state. Here is the D-equivalent of the above code:

    ⊃{                      ⍝ next item is ⍺
        (A B C)←⍵           ⍝ named items of tuple ⍵
        A∆←A ... ⍺          ⍝ new value for A depending on item ⍺
        C∆←C ... A∆ ... ⍺   ⍝ new value for C depending on A∆ and item ⍺
        ...                 ⍝ ...
        A∆ B C∆             ⍝ "successor" tuple (A and C changed)
    }/(⌽items),⊂A B C       ⍝ for each item and initial state tuple A B C

In this coding, the accumulating tuple arrives as the right argument (⍵) of the operand function, with the next “loop item” on the left (⍺). Notice how the items vector is reversed (⌽items) so that the items arrive in index-order in the right-to-left reduction.

If you prefer your accumulator to be on the left, you can replace the primitive reduction operator (/) with Phil Last’s foldl operator, which also presents its loop items in index-order:

    foldl←{⊃⍺⍺⍨/(⌽⍵),⊂⍺}    ⍝ fold left

then:

    A B C {                 ⍝ final state from initial state (left argument)
        (A B C)←⍺           ⍝ named items of tuple ⍺
        A∆←A ... ⍵          ⍝ new value for A depending on item ⍵
        C∆←C ... A∆ ... ⍵   ⍝ new value for C depending on A∆ and item ⍵
        A∆ B C∆             ⍝ successor tuple (A and C changed)
    } foldl items

If the number of elements in the state tuple is large, it can become unwieldy to name each on entry and exit to the operand function. In this case it simplifies the code to name the indices of the tuple vector, together with an at operator to denote the items of its successor:

    T ← (...) (...) ...         ⍝ intitial state tuple T
    A B C D ... ← ⍳⍴T           ⍝ A B C D ... are tuple item "names"

    T {                         ⍝ final state from initial state (left argument)
        A∆←(A⊃⍺) ... ⍵          ⍝ new value for A depending on item ⍵
        C∆←(C⊃⍺) ... A∆ ... ⍵   ⍝ new value for C depending on A∆ and item ⍵
        A∆ C∆(⊣at A C)⍺         ⍝ successor tuple (A and C changed)
    } foldl items

where:

    at←{A⊣A[⍵⍵]←⍺ ⍺⍺(A←⍵)[⍵⍵]}   ⍝ (⍺ ⍺⍺ ⍵) at ⍵⍵ in ⍵

There is some discussion about providing a primitive operator @ for at in Dyalog V16.

Examples
Function kk illustrates naming tuple elements (S E K Q) at the start of the operand function.
Function scc accesses tuple elements using named indices (C L X x S) and an at operator.

Tuples: a postscript
We might define a “tuple” in APL as a vector in which we think of each item as having a name, rather than an index position.

    bob         ⍝ Tuple: name gender age
┌───┬─┬──┐
│Bob│M│39│
└───┴─┴──┘

    folk        ⍝ Vector of tuples
┌──────────┬────────────┬──────────┬────────────┬─
│┌───┬─┬──┐│┌─────┬─┬──┐│┌───┬─┬──┐│┌─────┬─┬──┐│
││Bob│M│39│││Carol│F│31│││Ted│M│31│││Alice│F│32││ ...
│└───┴─┴──┘│└─────┴─┴──┘│└───┴─┴──┘│└─────┴─┴──┘│
└──────────┴────────────┴──────────┴────────────┴─

    ↓⍉↑ folk    ⍝ Tuple of vectors
┌─────────────────────────┬────────┬───────────────┐
│┌───┬─────┬───┬─────┬─   │MFMF ...│39 31 31 32 ...│
││Bob│Carol│Ted│Alice│ ...│        │               │
│└───┴─────┴───┴─────┴─   │        │               │
└─────────────────────────┴────────┴───────────────┘

    ⍪∘↑¨ ↓⍉↑ folk   ⍝ Tuple of matrices
┌─────┬─┬──┐
│Bob  │M│39│
│Carol│F│31│
│Ted  │M│31│
│Alice│F│32│
 ...   . ..
└─────┴─┴──┘

APLers sometimes talk about “inverted files”. In this sense, a “regular” file is a vector-of-tuples and an inverted file (or more recently: “column store”) is a tuple-of-vectors (or matrices).

The Halting Problem Rendered in APL

The halting problem is the problem of determining, from a description of a program and an input, whether the program will finish running or continue to run forever. It was proved in the negative by Alonzo Church and Alan Turing in 1936, and can be rendered in Dyalog APL as follows:

Suppose there is an operator H such that f H ⍵ is 1 or 0 according to whether f ⍵ halts. Consider:

In dfns

   f←{∇ H ⍵:∇ ⍵ ⋄ ⍬}

For f 0, if ∇ H 0 is 1, then ∇ 0 is invoked, leading to an infinite recursion, therefore ∇ H 0 should have been 0. On the other hand, if ∇ H 0 is 0, then the part is invoked and is the result of f 0, therefore ∇ H 0 should have been 1.

Presented as a table: For f 0

suppose ∇ H 0 is invokes consequence ∇ H 0 should be
1 ∇ 0 infinite recursion 0
0 f 0 results in 1

Therefore, there cannot be such an operator H.

In Tradfns

   ⎕fx 'f x' '→f H x'

For f 0

suppose f H 0 is invokes consequence f H 0 should be
1 →1     infinite loop 0
0 →0 f 0 exits 1

Rencontres Dyalog APL 2016 – Paris, France

It is really good to see APL events come back to life! In April of 2014, we witnessed the re-birth of SWEDAPL, which had been dormant for some time – but now meets twice a year and is perhaps the most vibrant APL meeting on our circuit, with many young developers developing new products and features in APL. The last SWEDAPL meeting drew a substantial international crowd, including a number of Danes – and the next one, scheduled for April 1st, is making a guest appearance in Copenhagen, hosted by Simcorp A/S – a bit like the Tour de France :-)

paris16This year we are really happy to be back in France, which has also had a relatively dormant APL community for the last decade or so – at least in terms of holding meetings. In this case we decided to arrange a meeting with the help of our French distributor – Quantys. Although the invitation was to a “Dyalog User Meeting“, about half of the attendees were users of other APL systems than Dyalog APL – with a little luck this meeting will turn out to contain the seeds for a rebirth of an independent French APL group. Fingers crossed!

Everyone please note: If you want to organise a local APL event, and you invite a speaker from Dyalog, we will do everything we can to send one or two delegates to your meeting. The current “circuit” includes Finland, Germany and Sweden twice a year, France and the East Coast USA. We have been known to show up at the Bay Area Users’ Group from time to time, in Toronto, at J and kx meetings – and recently also at FunctionalConf in Bangalore.

After welcoming remarks from Marc Righetti of Quantys, Gitte talked about Dyalog’s commitment to ensure that APL is well-integrated with modern computing platforms and infrastructure, which is always in the throes of another revolution. The current movement towards cloud computing and the need for platform independence is no exception. The good news is that Dyalog is growing to meet the challenge; we expect to add another couple of heads this year and grow the company by another ten percent.

Dan Baronet is a native of Montreal, Canada. As one of our French-speaking team members, he ended up doing most of the heavy lifting, with presentations on the upcoming v15.0 release, and a recap of the recent language enhancements in version 14.0 – in particular, the rank and key operators and function trains. Nicolas Delcros also spoke in French on the subject of his most recent work on integrating the publishing capabilities of Adrian Smith’s NewLeaf tool to SharpPlot, under the name SharpLeaf. I was only allowed to interrupt the flow of French twice, first with a road map presentation and, in the afternoon, a brief introduction to Futures and Isolates.

At the end of an action-packed day, Quantys treated us all to Champagne and snacks – many thanks to Marc for running the show and taking good care of us. A single day was much too short a time to do justice to the last decade of Dyalog achievements – so we will have to be back more regularly!

FinnAPL Forest Seminar 2016

The view from the Sauna. Some people actually went in!

The view from the Sauna. Some people actually went in!

Finns probably have better reasons to look forward to spring more than most of us: not only does it get much easier to keep that hole in the ice open, it is time for the annual FinnAPL Forest Seminar!

This year, just under 20 of us gathered for two days (Thursday March 10th and Friday March 11th) at Hirvihaara Manor, about an hour north of Helsinki, to update each other on what we have been getting up to recently.

Thursday

After a warm welcome from Jouko Kangasniemi, Chairman of FinnAPL, Veli-Matti Jantunen from Statistics Finland kicked the proceedings off with a talk titled “The long way of an APL2 bigot to Dyalog world”, where he discussed features of recent versions of Dyalog APL, awarding some of them them varying numbers of thumbs up, declaring some to be irrelevant. A few were found to be flawed… We are hoping to talk him into a repeat at Dyalog’16 as this was a valuable and thought-provoking review!

I was on next – with the Spring 2016 version of the Dyalog Road Map. As should be confirmed by the slides, there is not a big change in direction. We are planning to increase headcount by another 10% this year and continue investing in the core interpreter technology, APL compilers, and tools to help you build applications on a growing number of platforms.

Ants on the left. Ray Cannon on the right

Ants on the left. Ray Cannon on the right

After lunch, Ray Cannon showed us how to “Build a better ant brain”, producing wonderful, coloured, animations with ants crawling all over the big screen, using MiServer 3.0 and a bit of JavaScript – running under Dyalog APL on a Raspberry Pi!

My technical keynote in the morning had included a demo of a very early prototype of a Python interface, which will allow APL users to tap in to Python libraries. I was, therefore, very interested in the next presentation by Esa Pursiheimo from the VTT Technical Research Centre of Finland – which gave us all an introduction to the Python language. There is no question that the Python community has built libraries that could be very useful to APLers (although I cannot say the language itself impressed me much :-) ).

The last presentation of the day, titled “Data Driven Documents” (aka “D3″), was also about using libraries written in other programming language to extend APL applications. In this case the language was JavaScript. Jouko Kangasniemi from the Confederation of Finnish Industries (EK) showed how he is generating JavaScript to call the popular D3 Graphics Library and publish charts that are relevant to Economic planners in Finland. A collection of animated charts created using this technology can be found at http://ek.fi/materiaalipankki/tietografiikka/talous/viikon-graafit/.

Since we were in Finland, the afternoon ended with a visit to a traditional “smoke sauna”, before we all scrubbed up for the banquet.

Cheers! From left to right (more or less): Antero Ranne, Gitte Christensen, Esa Lippu, Miika Rämä, Simo Kilponen, Jouko Kangasniemi, Heikki Viitamäki, Esa Pursiheimo, Olli Paavola, Kaarlo Reipas, Göran Koreneff, Morten Kromberg, Kimmo Kekäläinen, Veli-Matti Jantunen, Timo Korpela, Ray Cannon (Missing: Anssi Seppälä)

Cheers! From left to right (more or less): Antero Ranne, Gitte Christensen, Esa Lippu, Miika Rämä, Simo Kilponen, Jouko Kangasniemi, Heikki Viitamäki, Esa Pursiheimo, Olli Paavola, Kaarlo Reipas, Göran Koreneff, Morten Kromberg, Kimmo Kekäläinen, Veli-Matti Jantunen, Timo Korpela, Ray Cannon (Missing: Anssi Seppälä)

Friday

The first talk on Friday morning was perhaps the most interesting from my point of view: Antero Ranne of the Ilmarinen Mutual Pension Insurance Company: Parallel showed how he was able to speed up financial simulations by a factor of approximately 3 on his Intel i7-based laptop, using Futures and Isolates in Dyalog version 14.0. It is really good to see that domain experts wield this tool!

After coffee, Gitte presented the work that she had done to put APL on the map as an invited speaker at a recent conference on the history of information technology in the Nordic region. She also reminded us all that we will be celebrating the 50th anniversary of the first running APL system on November 27 (http://silvermapleweb.com/first-cleanspace/). At Dyalog ’16 on October 9th-13th in Glasgow, Scotland, Dyalog will set time aside to celebrate this anniversary in collaboration with the British APL Association. If you have a good story about ground-breaking work done in APL in the early days, please get in touch and discuss how you might contribute to the celebrations!

Once again, I found myself standing between the audience and lunch – fortunately there are enough juicy language features and interfaces coming in versions 15.0 and 16.0 and I did not have anyone walk out before I was done. I even had time to talk about a workspace that we added several years ago, after discovering that several members of the audience were unaware of it: The “loaddata” workspace, which contains functions to read and write Excel Spreadsheets, CSV files, XML and ODBC data sources. If you have not seen it yet, try loading it and take a look.

After another excellent lunch, Anssi Seppälä of Enease Oy wrapped up the formal part of the programme with a talk on an inverted vectorial database implemented in the J programing language, named JD. JD makes it straightforward to manage large timeseries containing records of power usage and the quality of electricity delivered to consumers, perform analyses and generate visualisations of the data.

Several of us continued discussing programming challenges, while drinking (STRONG!!!) Finnish coffee and eating the wonderful cakes that were provided all day by Hirvihaara Manor, before heading back home after another successful FinnAPL Forest Seminar – we look forward to the 2017 edition!

Gitte and I managed to get about 40 hours at home before boarding the next plane, heading for Paris for the first French Dyalog User Meeting in recent history. More about that coming soon!

PQA

PQA is an acronym for Performance Quality Assurance. We developed PQA to answer the questions, which APL primitives have slowed down from one build of the Dyalog interpreter to the next, and by how much? Currently, PQA consists of 13,659 benchmarks divided into 136 groups.





4

The Graphs

The graph above (and all other graphs in this article) plot the timing ratios between version 14.1 and the build of version 15.0 on 2016-01-22, the Windows 64-bit Unicode version. The data are the group geometric means. The horizontal axis are the groups sorted by the means. The vertical axis are logarithms of the means. The percent at the top (-17.1%) indicates that over all 136 groups, version 15.0 is 17.1% faster than version 14.1. The bottom of the graph indicates how many of the groups are faster (blue, 69.9%) and how many are more than 2% faster (89/136); and how many are slower (red, 30.1%) and how many are more than 2% slower (27/136).

For the graph above, the amount of blue is gratifying but the red is worrying. From past experience the more blue the better (of course) but any significant red is problematic; speed-ups will not excuse slow-downs.

PQA is run nightly when a new build of version 15.0 is available. Each PQA run takes about 12.5 hours on a dedicated machine. The graphs from the past month’s runs are presented on the strip on the right; the first of them is a shrunken version of the big graph at the beginning of the article.

Challenges

About two years ago, investigations into a user report of an interpreter slow-down underscored for us a “feature” of modern computer systems: An APL primitive can run slower even though the C source is changed only in seemingly inconsequential ways and indeed even unchanged. Moreover, the variability in measurements frequently exceeded the difference in timings between the two builds, and it was very difficult to establish any kind of pattern. It got to the point where even colleagues were doubting us — how hard can it be to run a few benchmarks?

  • The system is very noisy.
  • Cache is king.
  • The CPU acts like an interpreter.
  • Branching (in machine code) is slow.
  • Instruction counts don’t tell all.
  • The number of combinations of APL primitives and argument types is enormous.

What can be done about it?

  • Developed PQA.
  • Run PQA on a dedicated machine.
  • Make the system as quiet as possible, then make it quieter than that. :-)
  • Set the priority of the APL task to be High or Realtime.
  • The ⎕ai or ⎕ts clocks have insufficient resolution. Use QueryPerformanceCounteror equivalent. The QPC resolution varies but is typically around 0.5e¯6 seconds.
  • Run each benchmark expression enough times to consume a significant amount of time (50e¯3 seconds, say).
  • Repeat the previous step enough times to get a significant sample.

Despite all these, timing differences of less than 5% are not reliably reproducible.

Silver Bullets

Given the difficulties in obtaining accurate benchmarks and given the amount of time and energy required to investigate (phantom) slow-downs, there evolved among us the idea of “silver bullets”: We will actively work on speed-ups instead of waiting for slow-downs to bite.

The speed-ups to be worked on are chosen based on:

  • decades of experience in APL application development
  • decades of experience in APL design and implementation
  • APLMON snapshots provided by users
  • talking to users
  • interactions on the Dyalog Forums
  • running benchmarks on user applications
  • user presentations at Dyalog User Meetings
  • the speed-up is easy, while mindful of Einstein’s admonition against drilling lots of holes where the drilling is easy

We were gratified to receive reports from multiple users that their applications, without changes, sped up by more than 20% from 13.2 to 14.0. A user reported that using the key operator in 14.0 sped up a key part of their application by a factor of 17. Evidently some of the silver bullets found worthwhile targets.

We urge you, gentle reader, to send us APLMON snapshots of your applications, to improve the chance that future speed-ups will actually speed up your applications. APLMON is a profiling facility that indicates which APL primitives are used, how much they are used, the size and datatype of the arguments, and the proportion of the overall time of such usage. An APLMON snapshot preserves the secrecy of the application code.

We will talk about the new silver bullets for version 15.0 in another blog post. Watch this space.

Coverage

We look at benchmarks involving + as a dyadic function to give some idea of what’s being timed. There are 72 such expressions. Variables with the following names are involved:

      'xy'∘.,'bsildz'∘.,'0124'
┌───┬───┬───┬───┐
│xb0│xb1│xb2│xb4│
├───┼───┼───┼───┤
│xs0│xs1│xs2│xs4│
├───┼───┼───┼───┤
│xi0│xi1│xi2│xi4│
├───┼───┼───┼───┤
│xl0│xl1│xl2│xl4│
├───┼───┼───┼───┤
│xd0│xd1│xd2│xd4│
├───┼───┼───┼───┤
│xz0│xz1│xz2│xz4│
└───┴───┴───┴───┘
┌───┬───┬───┬───┐
│yb0│yb1│yb2│yb4│
├───┼───┼───┼───┤
│ys0│ys1│ys2│ys4│
├───┼───┼───┼───┤
│yi0│yi1│yi2│yi4│
├───┼───┼───┼───┤
│yl0│yl1│yl2│yl4│
├───┼───┼───┼───┤
│yd0│yd1│yd2│yd4│
├───┼───┼───┼───┤
│yz0│yz1│yz2│yz4│
└───┴───┴───┴───┘

x and y denote left or right argument; bsildz denote:

b
s
i
l
d
z
boolean
short integer
integer
long integer
double
complex
1 bit
1 byte
2 bytes
4 bytes
8 bytes
16 bytes

0124 denote the base-10 log of the vector lengths. (The lengths are 1, 10, 100, and 10,000.)

The bsil variables are added to the like-named variable with the same digit, thus:

xb0+yb0  xb0+ys0  xb0+yi0  xb0+yl0
xs0+yb0  xs0+ys0  xs0+yi0  xs0+yl0
xi0+yb0  xi0+ys0  xi0+yi0  xi0+yl0
xl0+yb0  xl0+ys0  xl0+yi0  xl0+yl0

xb1+yb1  xb1+ys1  xb1+yi1  xb1+yl1
xs1+yb1  xs1+ys1  xs1+yi1  xs1+yl1
...
xi4+yb4  xi4+ys4  xi4+yi4  xi1+yl4
xl4+yb4  xl4+ys4  xl4+yi4  xl4+yl4

There are the following 8 expressions to complete the list:

xd0+yd0  xd1+yd1  xd2+yd2  xd4+yd4
xz0+yz0  xz1+yz1  xz2+yz2  xz4+yz4

The idea is to measure the performance on small as well as large arguments, and on arguments with different datatypes, because (for all we know) different code can be involved in implementing the different combinations.

After looking at the expressions involving+, the wonder is not why there are as many as 13,600 benchmarks but why there are so few. If the same combinations are applied to other primitives then for inner product alone there should be 38,088 benchmarks. (23×23×72, 23 primitive scalar dyadic functions, 72 argument combinations.) The sharp-eyed reader may have noticed that the coverage for + already should include more combinations:

  xb0+yb1  xb0+yb2  xb0+yb4
  xb0+ys1  xb0+ys2  xb0+ys4
  xb0+yi1  xb0+yi2  xb0+yi4
  xb0+yl1  xb0+yl2  xb0+yl4
  ...
  xl0+yi1  xl0+yi2  xl0+yi4
  xl0+yl1  xl0+yl2  xl0+yl4 

We will be looking to fill in these gaps.

More on the Graphs

You should know that the PQA graphs shown here are atypical. The “blue mountain” is higher and wider than we are used to seeing, and it is unprecedented for a graph (the one for 2016-02-26) to have no red at all.

From version 15.0, the list of new silver bullets is somewhat longer than usual, but silver bullets usually only explain the blue peak. (Usually, speed-ups are implemented only if they offer a factor of 2 or greater improvement, typically achieved only on larger arguments.) What of the part from the “inflection point” around 30 and thence rightward into the red pit?

We believe the differences are due mainly to using a later release of the C compiler, Visual Studio 2015 in 15.0 vs. Visual Studio 2005 in 14.1. The new C compiler better exploits vector and parallel instructions, and that can make a dramatic difference in the performance of some APL primitives without any change to the source code. For example:

      x←?600 800⍴0
      y←?800 700⍴0
      cmpx 'x+.×y' ⍝ 15.0
1.54E¯1
      cmpx 'x+.×y' ⍝ 14.1
5.11E¯1

(Not all codings of inner product would benefit from these compiler techniques. The way Dyalog coded it does, though.)

Together with improvements for specific primitives such as +.× the new C compiler also provides a measurable general speed-up. Unfortunately, along with optimizations the new C compiler also comes with pessimizations. (Remember: speed-ups will not excuse slow-downs.) Some commonly and widely used C facilities and utilities did not perform as well as before. Your faithful Dyalog development team has been addressing these shortcomings and successfully working around them. Having a tool like PQA has been invaluable in the process.

It has been an … interesting month for PQA from 2016-01-22 to 2016-02-27. Now it’s onward to ameliorating the red under Linux and AIX.