A Speed-Up Story

The first e-mail of the work week came from Nicolas Delcros. He wondered whether anything clever can be done with ∘.≡ on enclosed character strings. I immediately thought of using “magic functions“, an implementation technique whereby interpreter facilities are coded as dfns. I thought of magic functions because the APL expressions involved in this case are particularly terse:

   t ←' zero one two three four five six seven eight nine'
   t,←' zéro un deux trois quatre cinq six sept huit neuf'
   t,←' zero eins zwei drei vier fünf sechs sieben acht neun'
   b←⌽a←1↓¨(' '=t)⊂t

   cmpx 'a∘.≡a' '∘.=⍨⍳⍨a'
  a∘.≡a   → 9.69E¯5 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  ∘.=⍨⍳⍨a → 8.21E¯6 | -92% ⎕⎕
   cmpx 'a∘.≡b' '(a⍳a)∘.=(a⍳b)'
  a∘.≡b         → 9.70E¯5 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  (a⍳a)∘.=(a⍳b) → 1.43E¯5 | -86% ⎕⎕⎕⎕

   y←⌽x←300⍴a

   cmpx 'x∘.≡x' '∘.=⍨⍳⍨x'
  x∘.≡x   → 9.53E¯3 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  ∘.=⍨⍳⍨x → 1.52E¯4 | -99% ⎕
   cmpx 'x∘.≡y' '(x⍳x)∘.=(x⍳y)'
  x∘.≡y         → 9.55E¯3 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  (x⍳x)∘.=(x⍳y) → 1.95E¯4 | -98% ⎕

The advantage will be even greater if/when we speed up . (And, obviously, the idea is also applicable to ∘.≢ : just replace with and = with .)

Jay Foad objected that the comparisons above aren’t quite fair as the more verbose expressions should check that either ⎕ct←0 or that the arguments do not contain any elements subject to tolerant comparison. I countered that the checking in C would not greatly affect the benchmarks as the time to do the checking is O(m+n) but the benefits are O(m×n) .

Since the factors are so promising and the coding relatively easy, I went ahead and did the work, with the following results:

14.1 14.0 ratio cost of checking
a∘.≡a 9.16e¯6 9.69e¯5 10.58 1.09
a∘.≡b 1.53e¯5 9.70e¯5 6.34 1.08
x∘.≡x 1.48e¯4 9.53e¯3 64.39 1.00
x∘.≡y 1.94e¯4 9.55e¯3 49.23 1.01

The last column (the cost of checking that tolerant comparison is not involved) is under 10% and decreases as the argument sizes increase.

This work is another illustration of the ubiquity and practical usefulness of the “selfie” concept — in the new (14.1) implementation, x∘.≡y is faster when the left and right arguments are the same than when they are not. In particular, the selfie x⍳x or ⍳⍨x occurs twice, and bolsters the point made in item 3 of Sixteen APL Amuse-Bouches:

x⍳x are like ID numbers; questions of identity on x can often be answered more efficiently on x⍳x than on x itself.

Finally, after all that, I recommend that one should consider using x⍳y before using x∘.≡y . The latter requires O(m×n) space and O(m×n) time, and is inherently inefficient.

Creating Shortcuts (Microsoft Windows)

At Dyalog, a developer not only needs access to all of the readily available editions of the interpreter but also to earlier versions that are no longer officially supported. On my Microsoft Windows Desktop I have a folder that contains a shortcut to my developer builds of all these interpreters.

I’ve recently suffered a complete failure of my C drive which, of course, contained my Windows desktop and all my shortcuts – which were lost.

The Dyalog interpreter exists in Classic and Unicode editions and 32 and 64 bit flavours…for versions from 12.0 to 14.1 this is a total of 28 interpreters. I couldn’t bring myself to create each of those 28 shortcuts by hand; let’s not even consider that if I were to build both debug and optimised binaries that would be 56 shortcuts!

Fortunately we have an old COM example (in the shortcut.dws workspace) which is shipped with the product. It’s a trivial exercise to call the supplied SHORTCUT function in one or more (ahem!) loops to create all the required shortcuts:

 Dyalogs;vers;chars;bits;location;v;c;b;dyalog;dir;name

 vers←'12.0.dss' '12.1.dss' '13.0.dss' '13.1.dss' '13.2.dss' '14.0.dss' 'trunk'
 chars←'classic' 'unicode'
 bits←'32' '64'

 ⎕NA'u shell32|SHGetFolderPath* u u u u >0T'
 location←'\Dyalog',⍨2⊃SHGetFolderPath 0 0 0 0 255

 :For v :In vers
     :For c :In chars
         :For b :In bits

             dir←'e:\obj\',v,'\2005\apl\win\',b,'\',c,'\winapi\dev\dbg'
             dyalog←dir,'\dyalog.exe'
             name←'.lnk',⍨location,'\',({v↓⍨('.dss'≡¯4↑v)/¯4}v),' ',b,' ',c

             name SHORTCUT dyalog dir'' 1('' 0)0 ''
         :End
     :End
 :End

shortcuts