{"id":4057,"date":"2018-05-23T07:58:07","date_gmt":"2018-05-23T07:58:07","guid":{"rendered":"http:\/\/www.dyalog.com\/blog\/?p=4057"},"modified":"2018-05-23T08:20:09","modified_gmt":"2018-05-23T08:20:09","slug":"is-it-sorted","status":"publish","type":"post","link":"https:\/\/www.dyalog.com\/blog\/2018\/05\/is-it-sorted\/","title":{"rendered":"Is it Sorted?"},"content":{"rendered":"<h2>Motivation<\/h2>\n<p>I have been working on the Dyalog APL quicksort implementation. The following programming puzzle arose in the process of doing the QA for this work.<\/p>\n<div style=\"margin: 15px 90px 15px 90px;\"><code class=\"language-apl\">\u2375<\/code> is a simple array. Write a function <code class=\"language-apl\">sorted<\/code>, without using <code class=\"language-apl\">\u234b<\/code> or <code class=\"language-apl\">\u2352<\/code>, such that <code class=\"language-apl\">sorted \u2375<\/code> is 1 if <code class=\"language-apl\">\u2375<\/code> is sorted in ascending order and 0 otherwise.<\/div>\n<p>The point about not using <em>grade<\/em> is that this is supposed to be an independent check that <i>grade<\/i> is correct (remains correct) after the new work.<\/p>\n<h2>Real Vectors<\/h2>\n<p>The simplest case is when <code class=\"language-apl\">\u2375<\/code> is a numeric vector. If furthermore <code class=\"language-apl\">\u2375<\/code> are not complex numbers (a case addressed later), then<\/p>\n<pre><code class=\"language-apl\">   \u2227\/ 2\u2264\/\u2375\r\n<\/code><\/pre>\n<p>each item being less than or equal to the next one, checks that <code class=\"language-apl\">\u2375<\/code> is sorted. Since <code class=\"language-apl\">\u234b<\/code> uses exact comparisons, here we must set <code class=\"language-apl\">\u2395ct\u2190\u2395dct\u21900<\/code>. Morever, in order that decimal floating-point numbers (DECFs) be compared correctly, here <code class=\"language-apl\">\u2395fr\u21901287<\/code>.<\/p>\n<h2>Real Arrays<\/h2>\n<p>More generally, when <code class=\"language-apl\">\u2375<\/code> is a non-complex numeric matrix, we must check that each row precedes or is equal to the next row. If <code class=\"language-apl\">c<\/code> and <code class=\"language-apl\">d<\/code> are consecutive rows, then corresponding items are compared and at the first item where they differ, <code class=\"language-apl\">c[i]<\/code> must be less than <code class=\"language-apl\">d[i]<\/code>.<\/p>\n<pre><code class=\"language-apl\">   ~ 0 \u220a (2&gt;\u233f\u236a\u2375) \u2372 &lt;\\ 2\u2260\u233f\u236a\u2375\r\n<\/code><\/pre>\n<p>The expression incorporates two refinements:<\/p>\n<ul>\n<li>If <code class=\"language-apl\">\u2375<\/code> is not a matrix, first apply <code class=\"language-apl\">\u236a\u2375<\/code>.<\/li>\n<li>Instead of checking <code class=\"language-apl\">c[i]<\/code> is less than <code class=\"language-apl\">d[i]<\/code>, check that <code class=\"language-apl\">c[i]<\/code> is not greater than <code class=\"language-apl\">d[i]<\/code>. This finesses the case where <code class=\"language-apl\">c\u2261d<\/code> and there is no first item where they differ; that is, the case where <code class=\"language-apl\">&lt;\\2\u2260\u233f\u236a\u2375<\/code> is all 0s for that row.<\/li>\n<\/ul>\n<p><code class=\"language-apl\">&lt;\\<\/code>on a boolean vector has 0s after the first 1, (and is all 0 if there are no 1s). Therefore, <code class=\"language-apl\">&lt;\\2\u2260\u233f\u236a\u2375<\/code> finds the first item (if any) where one cell differs from the next cell, and that item must not be greater than the corresponding item in the next cell.<\/p>\n<p>For example:<\/p>\n<pre><code class=\"language-apl\">   x\u2190?97 3\u237410\r\n\r\n   {~ 0 \u220a (2&gt;\u233f\u236a\u2375) \u2372 &lt;\\ 2\u2260\u233f\u236a\u2375} x\r\n0\r\n   {~ 0 \u220a (2&gt;\u233f\u236a\u2375) \u2372 &lt;\\ 2\u2260\u233f\u236a\u2375} x[\u234bx;]\r\n1\r\n<\/code><\/pre>\n<p>(Muse: since <code class=\"language-apl\">x<\/code> above are random numbers, there is a possibility that it is sorted and the first test above can be 1. But: if each elementary particle in the visible universe were a computer and every nanosecond each of them creates a random matrix and tests it for sortedness as above, running from the beginning of the time to the end of our lives, it is still a <i>very<\/i> safe bet that no 1 would result.)<\/p>\n<p>For integer arrays, there is an alternative of using the signs of the arithmetic difference between successive cells:<\/p>\n<pre><code class=\"language-apl\">   {~ 0 \u220a 1\u2260t\u00d7&lt;\\0\u2260t\u2190 \u00d7 2-\u233f\u236a\u2375} x[\u234bx;]\r\n1\r\n<\/code><\/pre>\n<p>(The sign where consecutive cells first differ must not be 1.) However, computing the difference on floating point numbers can founder on overflow:<\/p>\n<pre><code class=\"language-apl\">   \u22a2 x\u2190\u00af1 1\u00d7\u230a\/\u236c\r\n\u00af1.79769E308 1.79769E308\r\n\r\n   {~ 0 \u220a 1\u2260t\u00d7&lt;\\0\u2260t\u2190 \u00d7 2-\u233f\u236a\u2375} x\r\nDOMAIN ERROR\r\n   {~0\u220a1\u2260t\u00d7&lt;\\0\u2260t\u2190\u00d72-\u233f\u236a\u2375}x\r\n                   \u2227\r\n<\/code><\/pre>\n<h2>Complex Numbers<\/h2>\n<p>Two complex numbers are ordered first by the real parts and then by the imaginary parts. (This is part of the TAO extension implemented in Dyalog APL version 17.0.) Therefore, a complex array can be tested for sortedness by testing an equivalent real array with each number replaced by their real and imaginary parts, thus:<\/p>\n<pre><code class=\"language-apl\">   (\u00af1\u233d\u23731+\u2374\u2374\u2375) \u2349 9 11\u2218.\u25cb\u2375\r\n   \u21919 11\u2218\u25cb\u00a8\u2375\r\n   9 11\u25cb\u23641 0\u22a2\u2375\r\n<\/code><\/pre>\n<p>Although the second expression is the shortest, it is less efficient in time, space, and number of <code class=\"language-apl\">getspace<\/code> calls. The last expression is favored for its brevity and performance.<\/p>\n<p>The number of <code class=\"language-apl\">getspace<\/code> is a worthwhile measure. Part of the QA process is a rather stringent procedure called the &#8220;Shuffle QA&#8221;. The entire Shuffle QA takes several weeks to run and its running time is directly related to the number of <code class=\"language-apl\">getspace<\/code>.<\/p>\n<h2>Character Arrays<\/h2>\n<p>None of the functions <code class=\"language-apl\">&lt; \u2264 \u2265 &gt; - \u00d7<\/code> are permitted on characters. This is solved by application of <code class=\"language-apl\">\u2395ucs<\/code>, converting characters to integers while preserving the ordering.<\/p>\n<h2>Putting It All Together<\/h2>\n<pre><code class=\"language-apl\">sorted\u2190{\r\n  \u2395ct\u2190\u2395dct\u21900\r\n  \u2395fr\u21901287\r\n  d\u219010|\u2395dr \u2375\r\n  d\u220a0 2: \u2207 \u2395ucs \u2375\r\n  d=9:   \u2207 9 11\u25cb\u23641 0\u22a2\u2375\r\n  ~ 0 \u220a (2&gt;\u233f\u236a\u2375) \u2372 &lt;\\ 2\u2260\u233f\u236a\u2375\r\n}\r\n<\/code><\/pre>\n<h2>Other Considerations<\/h2>\n<p>That <code class=\"language-apl\">\u2375\u2337\u2368\u2282\u234b\u2375<\/code> is sorted is a necessary but not sufficient condition that <code class=\"language-apl\">\u234b\u2375<\/code> is correct. For example, an &#8220;adversary&#8221; can supply the following results for <code class=\"language-apl\">\u234b\u2375<\/code> so that <code class=\"language-apl\">\u2375\u2337\u2368\u2282\u234b\u2375<\/code> is sorted:<\/p>\n<pre><code class=\"language-apl\">?\u2262\u2375\r\n(\u2262\u2375)\u2374?\u2262\u2375\r\n\u00af1\u2193\u234b\u2375\r\n\u220a i {\u2282\u2375[?\u2368\u2262\u2375]}\u2338\u2368 \u2375\u2337\u2368\u2282i\u2190\u234b\u2375\r\n<\/code><\/pre>\n<p>The last expression randomly permutes the grade indices of equal cells, a result which violates the requirement that <em>grade<\/em> indices of equal cells are in ascending order. That is, <em>grade<\/em> must be stable.<\/p>\n<p>In Dyalog APL version 17.0, <em>grade<\/em> has been extended to work on non-simple arrays, the much discussed TAO, total array ordering. Checking that a non-simple array is sorted without using <em>grade<\/em> requires facilities discussed in the paper <em>TAO Axioms<\/em> and is beyond the scope of this note.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Motivation I have been working on the Dyalog APL quicksort implementation. The following programming puzzle arose in the process of doing the QA for this work. \u2375 is a simple array. Write a function sorted, without using \u234b or \u2352, &hellip; <a href=\"https:\/\/www.dyalog.com\/blog\/2018\/05\/is-it-sorted\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":13,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-4057","post","type-post","status-publish","format-standard","hentry","category-apl-language"],"_links":{"self":[{"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/posts\/4057","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/comments?post=4057"}],"version-history":[{"count":5,"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/posts\/4057\/revisions"}],"predecessor-version":[{"id":5097,"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/posts\/4057\/revisions\/5097"}],"wp:attachment":[{"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/media?parent=4057"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/categories?post=4057"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dyalog.com\/blog\/wp-json\/wp\/v2\/tags?post=4057"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}