Planet Lisp

Common Lisp TipsThe return value of read-sequence

· 3 days ago

I’ve seen this idiom a few times when working with a binary stream:

(let ((elements-read (read-sequence buffer stream)))
  ...)

While the return value is the number of elements read in that situation, it’s a useful coincidence in a common case. The actual return value of read-sequence is:

the index of the first element of sequence that was not updated

Therefore, treating the return value as the number of elements read is completely wrong if you use the start option to read-sequence.

For example, if stream has one or more elements available to read:

(read-sequence buffer stream :start 42 :end 43)
=> 43

The actual number of elements read is 1, not 43.

Nicolas HafnerIt's Been So Long - Confession 71

· 9 days ago

header
It's been a good while since I last wrote an entry. I didn't write anything all January for a multitude of reasons, at the forefront being that it was exam season again. That's over with now, fortunately, so I do have some more time on my hands to goof off. Unfortunately though, I only have one more week left of this precious "free" time before university strikes me in the back again. I best use it wisely.

The first week of my holidays now I've spent on a heavy reworking of Portacle, a portable, multiplatform, upgradable, install-less Common Lisp development environment project. I've only just now finally got it to work properly on all three platforms and differing versions thereof. It has been an unbelievable struggle, as every platform posed unique problems and nasty edge-cases that were hard to catch without testing everything on a multitude of virtual machines all the time. If you look through the commit history you can find plentyful examples of the borderline insanity that I reached at points. I might write an in-depth article about my adventures another time.

During the exam session itself I was mostly occupied with working on Lichat, my own light-weight chat protocol. It has the nice properties that it is very uniform and rather simple to implement a client for. Among the nice aspects of the protocol are that everything goes over a channel, even private messages, that each update is associated with an ID and failures or responses can be tracked even if they happen to be out of order, and that connection multiplexing is provided directly by the protocol, so you can log into the same user account from multiple machines or instances simultaneously. I've written both a TCP and a WebSockets server, and a CL and JS client for it as well, so you can easily set up your own distributions of it. These CL and JS clients also have respective user interfaces as well, namely LionChat and Lichat-JS. Lionchat might turn into a generic chat client at some point in the future.

I've also continued work on Maiden, and finished a Lichat client for it as well. It's now running as a bot alongside the previous Colleen and I'm intending on switching over fully soon. The only thing that I haven't switched over yet is the chat logging, as I haven't had time to test that extensively yet. I've also been using its Text To Speech module to provide some fun interactivity for my video game streams. Before I can unleash Maiden onto Quicklisp I'll need to heavily document everything though, as it is quite a lot more complex -and capable- than Colleen was.

Then, I've also taken the time to switch over TyNET to the new Radiance 1.0. Radiance is now complete and ready for release as far as I'm aware. I've been holding off on releasing it for good as I've been waiting for some feedback on its tutorial from a couple of friends of mine. Once they get ahead with that I'll type up a good, long entry to announce everything. If my paper for ELS 2017 gets accepted, I will even be presenting it at the symposium in April. Speaking of the symposium- I've been working on a new website for it on and off, to which it will be switching in a while. You can get a preview peek at it here.

For Christmas I gave my father a Raspberry Pi 3 and installed Emacs and CCL onto it. He's been using it, and a DAQ board, to interface with a Korg synthesizer, since he's interested in making music on a more low, analog level. To do this I've been trying to teach Lisp to him since Christmas. While it's been a hard road so far, he's slowly getting into it and making advances. Given that he's been a Fortran programmer for almost all his life, with probably close to forty years of experience or more in it, I'm actually surprised at how well he's been doing with something that probably seems very different and alien to what he's used to. As part of this, though, I've had to write a few libraries, namely cl-gpio to interface with the Pi's GPIO pins, cl-spidev to use the serial port interface, cl-k8055 for the DAQ kit he's been using, and pi-plates for the pi-plates extension boards. The last library isn't done yet as I haven't had the time to test it yet, but the other three should work just fine.

So, I've been slowly, but surely, working off my long stack of projects that need to be completed someday. With Maiden out of the way soon too, I'll finally be able to get back to two of the remaining major projects: Trial, my game engine, and Parasol, my painting application. I'm still undecided as to which I'll tackle first, though. If you have a preference, let me know!

Finally, aside from programming work, I've gotten back into the swing of drawing. Before I was constantly feeling down and unmotivated about it. Everything just seemed to suck and I could never come up with ideas for things that I actually wanted to draw. I'm still not exploding with ideas now, but at least I've found something that inspires me a lot. Specifically, I was reading through Yokohama Shopping Trip and it was filled with exactly the kind of scenery-heavy, romantic drawings that I always wanted to make. "Romantic" referring to the Romanticism art movement. For the past few days then I've been trying to get a scenery drawing done every evening. You can see them on my tumblr. I still have a very long way to go and I'm painfully aware of all the things I do badly at or can't figure out how to draw properly, but I just have to keep on keeping on. It's too late to throw it all to the wind now.

In the near future I'd like to focus some more on writing things again, and perhaps also on actually reading some of the books that I've started accumulating over the years. But then, I'd also like to continue streaming video games, as I've been enjoying that quite a lot. Ah, there's just so much I'd like to do. If only someone were to figure out the age-old problem of finite capacity. Or in the very least: if only I didn't have to worry about earning any money to make a living, I could just merrily focus on being as productive as I can be instead of having to focus on what's marketable. Ah well, you can't have everything, eh?

So, in the more immediate future you can, if you want to, look forward to a full release announcement of Portacle, Radiance, and Maiden, more drawings, and hopefully more articles and stories as well. No promises, though. I'm not very good at keeping to deadlines for projects.

Zach BeaneGary Byers is stepping back from CCL and Clozure

· 11 days ago

Here’s a message from Andrew Shalit to openmcl-devel:

To the CCL Community -

As many of you know, Gary Byers has been on a leave of absence from his Clozure CL work to attend to his health. I'm writing to let you know that, unfortunately, this leave will be permanent. Gary has resigned from his position at Clozure and will no longer be the lead developer of Clozure CL.

Over the last 30+ years Gary has made a tremendous contribution to the Common Lisp programming community, first through his work on Macintosh Common Lisp, and then through the creation of CCL. Gary has always been ready to share his deep knowledge of the Common Lisp standard and the issues attending its implementation. I don't know anyone whose understanding of the details of Common Lisp could match Gary's. His many contributions will be missed.

The work on CCL will continue, though. In Gary's absence, Matthew Emerson will be stepping up as the primary maintainer of CCL. Matthew is moving CCL to GitHub, and he hopes that others will pitch in to continue the development of this great software platform. Clozure Associates will also continue to support CCL, through direct contributions and through our consulting work. I'll be supporting those efforts, and I hope you'll consider supporting them as well.

Yours,

Andrew Shalit
Clozure Associates

Eugene ZaikonnikovAnnouncing CL-VIDEO

· 13 days ago

I'm happy to announce CL-VIDEO, a basic AVI/MJPEG video decoder written in Common Lisp. The library leverages CL-JPEG for frame processing and CL-RIFF for container format handling.

The code has only been lightly tested on SBCL 13.x/Linux x86-64. Some sample files can be found here (the toy plane AVI) and here.

To run it, CL-JPEG version 2.7 is required. Since it's not in Quicklisp yet as of time of writing, it must be cloned into your local-projects. Then you can load cl-video-player system and run (cl-video::play <your filename.avi>).

The implementation is still very much naive, as expected from a weekend project. There is no support for indexing, and a number of edge cases of legit AVI encodings might fail. The library will decode video frames in the order they occur. No parsing of BITMAPINFOHEADER structures; the assumption is they are MJPG DIB. No audio playback, although the stream is being read and adding at least PCM/WAV playback shouldn't be too big an effort.The decoder is factored out into independent implementation in cl-video.lisp. A primitive CLX media player is included in player.lisp. Each AVI instance contains one or more (audio or video) stream record objects, which hold ring buffers of frames with read and write cursors. The interaction between the decoder and player frontend runnning in separate threads is done via mutexed access to this ring buffer. The player thread can be kicked off a callback supplied to decode method: it will be called once the header part is parsed.

Since CL-JPEG doesn't use any SIMD extensions, the performance is modest. On my 6 year old 3GHz i5 (running one core of course) it decodes 480p 25fps file without hiccups, but going beyond that would require implementing multicore support in decode. Still it might be useful as is in applications not requiring real-time decoding performance.

Quicklisp newsJanuary 2017 Quicklisp dist update now available

· 25 days ago
New projects:
  • ahungry-fleece — A general utility library of convenience functions and features. — GPLv3
  • asd-generator — Automatic directory scanner/generator for .asd project files. — GPLv3
  • cl-cache-tables — A wrapper around native hash-tables to facilitate in-process caching of common lisp data structures. — MIT
  • cl-feedparser — Common Lisp universal feed parser — LLGPL
  • cl-gpio — A library for the Linux GPIO kernel module as used on hobby kits such as the Raspberry Pi — Artistic
  • cl-k8055 — Bindings to the k8055 DAQ hobby board. — Artistic
  • cl-online-learning — Online Machine Learning for Common Lisp — MIT Licence
  • cl-spidev — A library for the Linux SPIDEV kernel module as used on hobby kits such as the Raspberry Pi — Artistic
  • cl-xdg — freedesktop.org standards handling — GNU General Public License
  • cl4l — esoteric CL essentials — MIT
  • glisph — Glyph rendering engine using OpenGL shading language — MIT
  • lichat-protocol — The independent protocol part of Lichat. — Artistic
  • lichat-serverlib — Tools to help build a server using the lichat protocol. — Artistic
  • lichat-tcp-client — A simple TCP client implementation for lichat — Artistic
  • lichat-tcp-server — A simple TCP server implementation for lichat. — Artistic
  • lichat-ws-server — A simple WebSocket server implementation for lichat. — Artistic
  • lionchat — A GUI client for the Lichat protocol — Artistic
  • lisp-binary — Declare binary formats as structs and then read and write them. — GPLv3
  • omer-count — A script to assist in counting the time period between Pesach and Shavuot. — GPL V3
  • ook — A CL compiler and enviroment for literate Orangutans. — Public Domain
  • opticl-core — A library for representing and processing images — BSD
  • rate-monotonic — A periodic thread scheduler inspired by RTEMS. — GPL-v3
  • timer-wheel — A timer wheel implementation with BORDEAUX-THREADS backend. — MIT
  • translate-client — A client to online web-server translators, currently only google translate — MIT
  • trivial-escapes — C-style escape directives for Common Lisp. — Public Domain
Updated projectsacclimationalexaanaphoraarchitecture.service-providerasdf-system-connectionsasdf-vizbeastbinfixcaveman2-widgetscaveman2-widgets-bootstrapcl-anacl-ansi-termcl-arxiv-apicl-association-rulescl-autowrapcl-csvcl-cudacl-custom-hash-tablecl-dbicl-digraphcl-enumerationcl-freetype2cl-glfw3cl-gobject-introspectioncl-hamtcl-jpegcl-liballegrocl-libyamlcl-marshalcl-openglcl-opsresearchcl-permutationcl-qrencodecl-readlinecl-routescl-statsdcl-syslogcl-typesettingcl-unificationcl-waylandcl-yamlclazyclinchclmlclobbercloser-mopclssclxcroatoancurry-compose-reader-macrosdeedsdefenumdexadoresrapfare-quasiquotefare-scriptsfare-utilsfast-iofemlispfixedformat-string-buildergsllinquisitorjonathanlambda-readerlet-pluslisp-unitlocal-timelog4clmcclimmetabang-bindmitomk-string-metricsmodularizeneo4clnew-opningleoclclopticlparachuteparse-floatparser.common-rulespipingplumppostmodernpostmodernityprovepsychiqqtools-uiqueuesrestasretrospectiffrfc2388-binaryserapeumspinneretstatic-vectorsstumpwmsxqltriviatrivia.balland2006trivial-ldaptrivial-rfc-1123trivial-updateubiquitousuiopunix-optsvarjovgplotweblocks-utilsxhtmlgenzlib.

Removed projects: cl-ledger, weblocks-examples.

To get this update, use: (ql:update-dist "quicklisp")

Enjoy!

Eugene ZaikonnikovNew in CL-JPEG

· 32 days ago

In course of last few months there were numerous small changes introduced to CL-JPEG. None substantial enough to warrant own announcement, but taken together perhaps it's due for an update. So here we go, as of version 2.6:

  • Pre-allocated buffers in DECODE-IMAGE and DECODE-STREAM are now supported. This should help reduce consing in bulk-processing applications. The buffer can be pre-allocated based on dimensions via JPEG:ALLOCATE-BUFFER.
  • CMYK JPEG support. YCCK to CMYK conversion is now performed by decoder. To convert into 3-component RGB, use JPEG:CONVERT-CMYK-TO-RGB convenience function.
  • An option to disable colorspace conversion via :COLORSPACE-CONVERSION NIL supplied to decode functions has been added. Can be useful e.g. if one needs the luminance component. Support of the corresponding option for encoder to improve performance in transcoding applications is in the plans.
  • Small bugfixes and performance tweaks.

Zach BeaneCommon Lisp in the Wild - Deploying Common Lisp Applications.

· 32 days ago

Zach BeaneMKCL 1.1.10 is now available

· 32 days ago

LispjobsWeb Technologist, Cadence, San Jose, California

· 48 days ago

They’re looking for a “full stack Java programmer,” but must also have 3+ years hand-on experience in Lisp/Tk Programming on Linux.

See web technologist job posting.


Quicklisp newsThank you

· 50 days ago
Today (December 31) is the last day for the Quicklisp appreciation fundraiser. It has met and exceeded the matching fund goal by a wide margin. I am so thankful of all the support for this fundraiser, but also for the many kind words of appreciation, and for the "Quicklisp supporter club" donors, over the years. The positive feedback has made the work very satisfying.

Thank you, and best wishes for a happy 2017!

McCLIMProgress report #5

· 53 days ago

Dear Community,

During this iteration I have continued working on the tutorial, fixing the issues and assuring CLIM II specification compatibility.

The most notable change is standard-application-frame implementation refactor which simplifies the code and moves some computation to creation time (until now we have searched through sheet hierarchy at each iteration of command loop and at various other occasions at runtime).

The rest of the time was spent on peer review of the contributions, merging pull requests, development discussions, questions on IRC and other maintenance tasks.

Alessandro Serra has created a Framebuffer Backend - a working proof of concept that McCLIM may work on top of a frame buffer. Albeit a bit slow you may find instructions how to run it here:

https://github.com/robert-strandh/McCLIM/wiki/CLX-backend(s)

A detailed report is available at:

https://common-lisp.net/project/mcclim/static/documents/status-reports.org

If you have any questions, doubts or suggestions - please contact me either by email ([email protected]) or on IRC (my nick is jackdaniel).

Happy new year!

Sincerely yours,
Daniel Kochmański

ABCL DevAn Armed Bear in Hock with the Dock

· 53 days ago
This season finds us remembering Wilde's remark about fashion being so unbearable that it must be utterly destroyed every six months.  In that spirit, we submit to current fashion in documenting how ABCL repl may be obtained locally viz:

   docker run --tty easye/abcl /usr/local/bin/abcl

Happy holidays!


One may need to explicitly update the base openjdk8 container that easye/abcl is based on via:

        docker pull easye/openjdk8
        docker run easye/abcl /usr/local/bin/abcl

[Docker Engine]: https://www.docker.com/products/docker-engine

If you want a specific packaging to add your own Java/Lisp artifacts, you may easily rebuild the Docker image that the container starts with from cloning the ABCL source and then issuing:

        docker pull easye/openjdk8
        docker build -t easye/abcl .

See http://abcl.org/svn/trunk/abcl/Dockerfile  for build instructions.

Marco Antoniotti

· 54 days ago

Orangutans grok Common Lisp

This being the season to be jolly (and eat bananas), I produced a magical piece of software: a compiler and decompiler for the language Ook!  Which proves that orangutans grok Common Lisp.

You can try out this recreational and magical piece of software by downloading the library from common-lisp.net.  A brief introduction can be found, of course, in the library (if you can see it!)

Please send bananas and DO NOT say "monkey"!

Enjoy

(cheers)

Zach BeaneBringing back random-state.net

· 59 days ago

When I tried to find Nikodemus Siivola’s nice post about load-time-value and inline caches, I found his site, random-state.net, was no longer working.

At my request, he managed to get it up and running at a new address long enough for me to make a mirror of all his posts and articles.

Until the “real” location is fully restored, you can access the content of the site at random-state.xach.com, including the nice article about load-time-value and inline caches.

When random-state.net comes back, I’ll redirect everything back there.

Enjoy!

Zach BeaneMake FFI callabacks work properly on 64 bit Windows OSes. � sharplispers/cormanlisp@2156983

· 60 days ago
Make FFI callabacks work properly on 64 bit Windows OSes. � sharplispers/cormanlisp@2156983:

Making a project open source is no guarantee that anyone will care. But in the case of Corman Lisp, Artem Boldarev's hard work on fixing problems has not only revived the project, but revived Roger Corman's interest in it:

Artem, thanks so much for this fix!!!!! I believe this was the one the caused me to stop working on it for some time, and ultimately just open source it. I wasn't sure anyone else would be able (or willing) to track it down. I had dealt with and fixed many such issues during corman lisp development. However when I hit this issue specific to 64-bit OS (and those were fairly rare then, but of course now ubiquitous) I spent some evenings and just decided I couldn't do it any more. I got as far as noticing that the callback code broke and was getting a garbled address (I think an 8 byte address/4 byte address mismatch), and this probably happened when an access violation occurs (these are employed for garbage collection heap management). Now that I see it is working well on 64-bit systems I am inspired to get back into it.

Thank you, Roger, for sharing the source, and thank you, Artem, for diligently hacking on it!

Zach BeaneCHIP-8 in Common Lisp: The CPU

· 61 days ago

Zach Beanetamamu/glisph

· 62 days ago
tamamu/glisph:

I like the demo GIF!

ECL NewsECL 16.1.3 release

· 62 days ago

Announcement

Dear Community,

After almost a year of development we are proud to present a new release of ECL tagged with version 16.1.3. All changes are backward compatible fixing bugs and other issues, implementing new interfaces and cleaning up the code base.

ECL manual has been updated in a few places. Work on a new documentation is still pending. New documentation is still incomplete, but you may see it here: https://common-lisp.net/project/ecl/static/ecldoc/.

Before this release we have performed extensive tests on many platforms (Linux, FreeBSD, OpenBSD, NetBSD, OSX, Windows MSVC, Windows MinGW, Windows Cygwin, Android and Haiku). For details please consult https://gitlab.com/embeddable-common-lisp/ecl/issues/307. Extra attention has been paid to Windows testing to improve that platform support.

This release is available for download in form of a source archive:

Best regards,
ECL Development Team

API changes

  • Added better interface for package-locks.

    Introduced functions:

    ext:package-locked-p package
    ext:lock-package package
    ext:unlock-package package
    ext:without-package-locks &body body
    ext:with-unlocked-packages (&rest packages) &body body
    

    To use these functions user has to require the module

    (require '#:package-locks)
    

    defpackage accepts new option lock to allow locking package on creation:

    (defpackage foo (:lock t))
    
  • mp:holding-lock-p: introduce new function for multiprocessing. Function verifies if lock is hold by the thread which calls the function. Usage: (mp:holding-lock-p my-lock).

  • make-random-state: fix problem with simple-vectors. The correct initialization types for make-random-state are: (OR RANDOM-STATE FIXNUM (MEMBER T NIL)).

    Initializing a random state with an appropriate array (element type and arity dependent on platform) is also possible.

  • ext:random-state-array: new extension for random-states. Usage: (ext:random-state-array random-state).

  • ext:terminate-process: new extension for external processes. Usage: (ext:terminate-process process) with a second, optional boolean argument whenever termination should be forced or not.

Enhancements

  • Implemented CDR-7. https://common-lisp.net/project/cdr/document/7/index.html

  • implemented CDRs: CDR-1, CDR-5, CDR-14. Both CDR-1 and CDR-5 were already implemented, CDR-14 made us to list them in *features* (https://common-lisp.net/project/cdr/document/14/index.html).

  • if ECL is build with --with-cxx option, :CXX-CORE is present in *features*.

  • deprecated configure option --with-local-gmp has been removed - use --enable-gmp (defaults to auto).

  • configure options has been revised.

  • ASDF has been upgraded to version 3.1.7.26 (with a few patches scheduled for 3.2.0).

  • bundled CLX has been purged. Lately I've fixed ECL support on portable CLX maintained by sharplispers on https://github.com/sharplispers/clx (available via QuickLisp).

  • initial port for the Haiku platform. The port is done by Kacper Kasper's work, one of Haiku developers. Threads are not supported yet.

  • refactored ECL internal tests framework. Tests in src/tests are now asdf-loadable (with load-source-op) and divided into test suites. make check target runs all regression and feature tests which aren't supposed to fail.

  • removed 15000 lines of obsolete code. Files not included in the buildsystem but lingering in the codebase or options failing to build. All info is added in the new documentation in the section "Removed interfaces".

  • improved man page and help output. Man page now contains up-to-date list of flags, as well as explanation of flag's behavior.

  • deprecated long flags with one dash, added two-dash version. Flags that aren't one-character, but start with one dash (e.g. -eval) are now deprecated; long version --eval was added instead.

  • indented C/C++ code to follow emacs's gnu C style. This is a first step towards coding standards in the documentation. Additionally all in the src/c/ directory are listed in the appropraite documentation section (new-doc).

  • refactored list_current_directory in unixfsys.d. Function was obfuscated with ifdefs with non-even pairs of #\{ and #\}.

Issues fixed

  • ECL signals floating point exceptions in top-level console.

  • mp:rwlock is treated as built-in class (previously process crashed if class-of was called on such object).

  • ECL builds now succesfully with --with-ieee-fp=no option.

  • ext:file-stream-fd: doesn't cause an internal-error if called with something not being a file-stream (signals a SIMPLE-TYPE-ERROR condtition).

  • stable-sort: bugfix and improvement in speed. Adapted from SBCL by Diogo Franco.

  • typep: accept * type specifier as abbreviation of T as described in 2.4.3 Type Specifiers of the specification.

  • MOP: fix problemes when redefining non-standard and anonymous classes. Bugs identified and fixed by Pascal Costanza.

  • getcwd: fix issue with too long pathname. This fixes the regression, which crashed ECL at start when pathname exceeded 128 characters limit.

  • make-random-state: fix a problem with simple-vectors. Until now #$ reader macro accepted simple vectors as an argument, what lead to bugs if vector didn't match specific requirements like the element type or the arity. Now we sanitize this.

  • make-load-form: provide implementation for random-state objects.

  • thread fix on msvc: on windows importing thread was closing the thread handler so the thread wakeup wasn't working because the handler is not more valid.

  • import thread wasn't set upping a proper environment: on some case the thread was mistakenly thinking that the thread was already registered.

  • ECL_HANDLER_CASE and ECL_RESTART_CASE didn't work as expected. Bug identified and fixed by Vadim Penzin.

Zach BeaneCryptanalysis with Reasoning Systems

· 65 days ago

Marco Antoniotti(R)creational Common Lisp

· 65 days ago

In my real work, I (or better, my students and colleagues) use R to program our data analyses (if you are interested, you can check out the TRONCO package.)

Of course, being a Common Lisp quite-high priest at this point and a recreational programmer in it, I wondered how to render several R constructs in the language with parentheses.  It turns out that it was quite fun to re-engineer some of the key features of the R language.

This is not the first such project trying to mimic R.  I know at least two of them; the most public one being Common Lisp Stat, but, being known for suffering from NIH and RTW syndromes, I had to jump in.

Here is a short summary of what I did and what was achieved.

Generic Math, Arrays and Vectors

R provides generic math on vectors.  Pretty much everything in R is a vector (stored in column major order; more on this later) including arrays.  The generic math helps you writing things like
> 42 * 1:5 # The ':' operator produces a vector.
[1]  42  84 126 168 210

Indexing

The generic math and the construction of some operators in CL is pretty much straightforward.  Where things get hairy is in the handling of the several ways that a R programmer may use to index, slice and dice vectors and arrays. As an example, you could write
> a = array(11:14, dim = c(2, 2), dimnames = list(list("a", "b"), list("x", "y")))

> a
   x  y
a 11 13
b 12 14

In brief, you have a matrix with "named" rows and columns. There are other idiosyncrasies but this is already something interesting to deal with, as you can now write things like:
> a[1, "y"]
[1] 3

> a["b", ]
 x  y 
12 14

The second "indexing" selects, or slices, the entire second row (counting from 1) and, as a result, returns a vector with the column names retained.

Indexing can also become interesting because of other idiosyncrasies in R.  Consider the following:
> a[2]
[1] 12

> a[42]
[1] NA # 'NA' is the 'not available' indicator.

> a[c(2, 3, 4, 10)]
[1]  12  13  14 NA

Indexing by only one index accesses the underlying vector in column major order. If the index is beyond the current limit, the value NA ('not available') is returned.  If the sole index is a vector, then the indexing returns a vector of the values in the array accessed column major order.

Indexing and Assignments

Of course, once you handle slicing and dicing of arrays, you have to deal with setting these slices and dice.

In R you can have a lot of what you can dream of (and something more, some of which, you wished was not there).

Basic assignment on vectors and arrays works as expected. Consider:
> a = array(11:19, dim = c(3, 3), dimnames = list(list("a", "b", "c"), list("x", "y", "z")))

> a
   x  y  z
a 11 14 17
b 12 15 18
c 13 16 19

> a[2, "y"] = 42

> a
   x  y  z
a 11 14 17
b 12 42 18
c 13 16 19

So far, nothing special is happening. Things actually make sense when trying to set a sub-slice of the array.
> b = array(42, dim = c(2, 2))

> b
     [,1] [,2]
[1,]   42   42
[2,]   42   42

> a[2:3, 2:3] = b

> a
   x  y  z
a 11 14 17
b 12 42 42
c 13 42 42

However, when setting a sub-slice of an array to a vector, or setting the elements of an array using a single vector index (i.e., when accessing the array in column major order), things may become a bit surprising.
> a[2:3, 2:3] = c(1, 1, 1, 1)

> a # This is not a surprise.
   x  y  z
a 11 14 17
b 12  1  1
c 13  1  1

> a[2:3, 2:3] = c(1, 2, 3, 4, 5, 6) # This is also expected (modulo the 'multiple'.)
Error in a[2:3, 2:3] = c(1, 2, 3, 4, 5, 6) : 
  number of items to replace is not a multiple of replacement length

> a[c(1, 2, 6, 7)] = c(111, 111, 111, 111) # Ok, understandable.
> a
    x   y   z
a 111  14 111
b 111   1   1
c  13 111   1

> a[c(1, 2, 12, 7)] = c(1, 1, 1, 1)

> a # What?!?
 [1]   1   1  13  14   1 111   1   1   1  NA  NA   1

Ok, the last example shows how R manages assigning past the limit of the underlying vector.

Not really a problem, once you get the hang of it, but, source of a bit of confusion nevertheless, at least to yours truly.

In any case, all of this must be dealt with once you want to get into building some facilities like these on top of Common Lisp.

Let's "Reinvent the Wheel" (RTW) in Common Lisp

Getting the basics down was not so difficult.  Common Lisp has a somewhat richer set of array and vector data types than R, however, some design decisions resulted in some fun (albeit probably sub-optimal) setup.

Here is a list of decisions taken, which may be subject to debate.

  • Objects are represented as structures.
  • Constructors follow the R conventions (no make-array!)
  • Vectors and arrays are mapped to CL vectors and arrays.
    This means that
    • Arrays are kept as multi-dimensional entities which are accessed in row major order
    • Vector and array "objects" are separate and different.
  • A generic reference function AT is introduced (and its counterpart, (SETF AT).)

Of course, these choices will make porting code from R to the CL library/tool quite cumbersome; let's say that this goal was dropped early on. The CL library (which, by the way, is named ρ, that is: RHO) will look familiar to a R programmer, but it will not be the same. After all, is it Common Lisp, isn't it?

Here is a run down of the basic features.  This was the hard part, in the opinion of the author.  Getting more of R implemented will be somewhat easier.

The AT Generic Function

The AT generic function works as expected on the standard CL objects (as it should).
RHO 10 > (defvar qwerty "qwerty")
QWERTY

RHO 11 > (setf (at qwerty 3) #\R)
#\R

RHO 12 > qwerty
"qweRty"

RHO 13 > (at qwerty 4)
#\t

RHO 14 > (defvar a #2A((1 2 3) (4 5 6) (7 8 9)))
A

;;; ...

RHO 16 > (at a 1 1)
5

RHO 17 > (setf (at a 0 0) 42)
42

RHO 18 > a
#2A((42 2 3) (4 5 6) (7 8 9))

The generic function AT also works on structures and class instances.
RHO 22 > (defstruct foo a s d)
FOO

RHO 23 > (defvar foo (make-foo :s 42))
FOO

RHO 24 > (at foo 'foo-s) ; In this case the accessor function FOO-S is invoked.
42

RHO 25 > (setf (at foo 'foo-d) "Food")
"Food"

RHO 26 > foo
#S(FOO :A NIL :S 42 :D "Food")

So far, so good.  Now let's have some fun.

Creating Arrays (and Vectors and Matrices)

Let's create some arrays, vectors and matrices.
RHO 28 > (setf rr (range 1 9)) ; This would be 'rr = 1:8' in R.
  1  2  3  4  5  6  7  8


RHO 29 > (setf rv (vector 1 22/23 -3.2 42 5.8))
      1  22/23   -3.2     42    5.8

RHO 30 > (setf rv2 (vector* (list 1 22/23 -3.2 42 5.8)
                            :names '(a b c d e)))
      A      B      C      D      E
      1  22/23   -3.2     42    5.8

The objects returned by VECTOR and VECTOR* (the full constructor) are wrappers around the underlying CL vectors.  Their class is RHO:VECTOR.  Note also that *PRINT-READABLY* is set to NIL, therefore, a "pretty printing" of the vectors is produced.
RHO 37 > (setf ra (array (range 1 9) :dim '(3 3)))
      [, 0] [, 1] [, 2]
[0, ]     1     2     3
[1, ]     4     5     6
[2, ]     7     8     1

RHO 38 > (at ra 1 1)
5

The function RANGE excludes the upper limit (following CL conventions), while the ARRAY constructors "recycles" the values passed to it (following R conventions).  The value produces is pretty-printed, while its class is RHO:ARRAY.  Indexing is 0-based, as in CL.

Creating matrices is straightforward and, again, a R-like function is used.
RHO 3 > (setf rm (matrix '(1 1 2 0) :nrow 2 :ncol 2))
      [, 0] [, 1]
[0, ]     1     1
[1, ]     2     0

RHO 4 > (at rm 0 1)
1

RHO 5 > (at rm 1 0)
2

Slicing and Dicing in RHO

Now that we have all the bases covered, we can proceed to some "slicing and dicing".  All the R indexing and assignment methods are provided in RHO, modulo the Common Lisp background choices, bugs and unimplemented features; not many of the last kind.

First of all, let's set some dimension names...
RHO 7 > (setf (dimension-names ra) '(("a" "b") ("x" "y")))
(("a" "b") ("x" "y"))

RHO 8 > ra
      [, x] [, y] [, 2]
[a, ]     1     2     3
[b, ]     4     5     6
[2, ]     7     8     1

RHO 9 > (at ra 1 "y")
5

Next we create a new array (a matrix, but in R, matrices are separate subclasses of arrays) rb and use it as a value to set a sub-matrix of ra.
RHO 13 > (setf rb (array 42 :dim '(2 2)))
      [, 0] [, 1]
[0, ]    42    42
[1, ]    42    42

RHO 29 > (setf (at ra (vector 1 2) (vector 1 2)) rb)
#2A((42 42) (42 42))

RHO 15 > ra
      [, x] [, y] [, 2]
[a, ]     1     2     3
[b, ]     4    42    42
[2, ]     7    42    42

As you can see, everything works as expected.

Remember that we have built a matrix rm; let's use it to set some elements of the array ra, according to the R "access-by-matrix" semantics.
RHO 16 > rm
      [, 0] [, 1]
[0, ]     1     1
[1, ]     2     0

RHO 24 > (type-of rm)
MATRIX

RHO 27 > (setf (at ra rm) 1024)
1024

RHO 28 > ra
      [, x] [, y] [, 2]
[a, ]     1     2     3
[b, ]     4  1024    42
[2, ]  1024    42    42

RHO 29 > (setf (at ra #(1 2) #(1 2)) #(1 1 1 1))
#(1 1 1 1)

RHO 30 > ra
      [, x] [, y] [, 2]
[a, ]     1     2     3
[b, ]     4     1     1
[2, ]  1024     1     1

Note however, that, although "correct", no attempt to signal an error is made for incongruous dimensions.
RHO 42 > (setf (at ra #(1 2) #(1 2)) #(11 12 13 14 15))
#(11 12 13 14 15)

RHO 43 > ra
      [, x] [, y] [, 2]
[a, ]     1     2     3
[b, ]     4    11    12
[2, ]  1024    13    14

Getting the indexing of different combinations of scalars, vectors, and arrays was an interesting exercise. The implementation is probably bloated and not easily understandable; most probably because of the design choice of keeping the underlying CL vectors and arrays "as-they-are", especially multidimensional arrays.

Missing Things...

Many...
Most important one is "negative" indexing, which does not work completely yet.
Of course, I am limiting this comment to arrays and vectors: other R data structures, such as factors and data.frames are in the making, but all the heavy lifting is now done.

Want to Play?

Well, for the time being, drop me a line (you know where to find me).  We can talk about setting up a proper project and discuss some design choices.


Cheers

Zach BeaneLISP memory woes, but not

· 65 days ago

Zach BeaneQuicklisp fundraiser update

· 72 days ago

The Common Lisp Foundation Quicklisp fundraiser hit its initial goal of $6,000 after just a few hours. There have been many donors and a few very generous donations put it over the top. I am so surprised and grateful that it happend so quickly.

By exceeding $6,000, contributions are no longer doubled with matching funds. But the fundraiser is still going to continue to December 31st as scheduled, and everything raised until then is important and appreciated.

As I write this, contributions are near $9,700. With the matching funds, it’s nearly $15,700. How high can it go?

Thank you for supporting Quicklisp!

Zach BeaneThe Quicklisp fundraiser is now up and running

· 72 days ago

The Common Lisp Foundation’s Quicklisp appreciation fundraiser is now up and running. I made a brief video to appeal for your support. Here’s the transcript:

Hi, I’m Zach Beane and I’m asking for your support for Quicklisp.

I created Quicklisp in 2010. It’s easy to install and provides nearly fifteen hundred Common Lisp community libraries with just a few commands.

I keep things updated every month, and work with the community to get new projects added and make sure existing projects keep working. I check builds, I look for problems, and I’ve filed hundreds of bug reports to keep those problems from affecting Quicklisp users.

The vast majority of this work has been done on a volunteer basis, in my free time.

But today, the Common Lisp Foundation has started a appreciation fundraiser for Quicklisp, running until December thirty-first.

I hope to use the proceeds from the fundraiser for a few months of dedicated work on Quicklisp. In particular, I want to address the main issues that have kept it in beta all this time, starting with security and documentation.

If Quicklisp has made your life easier - whether it’s to get started as a new Common Lisp user, or to easily get all the dependencies you need in a large commercial project, or if it just helps you hack on a hobby or homework project - please consider visiting quicklisp.org/donate for a contribution.

Thank you!

Quicklisp newsDecember 2016 Quicklisp dist update now available

· 73 days ago
Note: The Quicklisp fundraiser is up and running. If you appreciate Quicklisp, please contribute if you can.

New projects:
  • cl-digraph — Simple directed graphs for Common Lisp. — MIT/X11
  • cl-directed-graph — Directed graph data structure — MIT
  • format-string-builder — A DSL wrapping cl:format's syntax with something more lispy. — MIT
  • hunchentools — Hunchentoot utility library — MIT
  • l-system — L-system or Lindenmayer system on lists — GPLv3+
  • parser.common-rules — Provides common parsing rules that are useful in many grammars. — MIT
  • parser.ini — Provides parsing of Ini expressions. — LLGPLv3
  • postmodernity — Utility library for the Common Lisp Postmodern library — MIT
  • stl — Load triangle data from binary stereolithography (STL) files. — ISC
  • whofields — HTML field rendering and input validation utilities written in Common Lisp — MIT
Updated projects3bmdalexaalexandriaarchitecture.builder-protocolarchitecture.hooksarchitecture.service-providerasdf-dependency-grovelbeastcarriercaveman2-widgetscellsceplcepl.drm-gbmcepl.sdl2cepl.skittercircular-streamscl+sslcl-anacl-autowrapcl-bootstrapcl-cacl-change-casecl-dbicl-dotcl-drmcl-gistscl-growlcl-jpegcl-kanrencl-l10ncl-libyamlcl-mediawikicl-mpg123cl-openglcl-out123cl-pangocl-portaudiocl-pslibcl-quickcheckcl-sdl2clackclavierclfswmclinchcltclcoleslawcollectorscroatoandbusdendritedjuladocumentation-utilsdynaeazy-gnuplotesrapfare-scriptsformletsgbbopengendlgenevagsllgtk-cffihttp-bodyhu.dwim.computed-classhu.dwim.defhu.dwim.presentationhu.dwim.serializerhu.dwim.syntax-sugarhyperluminal-meminquisitorjskenzolacklasslegitmcclimmetabang-bindmeteringmglmitomodularizemodularize-interfacesneo4clnibblesningleoclclopticlpgloaderpng-readpostmodernprotobufpsychiqqlotqt-libsqtoolsquickutilquriratifyretrospectiffrutilsserapeumskitterspinneretstaplestumpwmsxqltriviatrivial-documentationtrivial-featurestrivial-nntptrivial-rfc-1123trivial-yencubiquitousufoutilities.print-treeutility-argumentsutils-ktvarjowebsocket-driverwhat3wordswookieworkout-timerxml.location.

Removed projects: cl-xspf, date-calc, elephant, html-entities, lambda-gtk, perfpiece, quicksearch, usocket-udp.

To get this update, use (ql:update-dist "quicklisp"). Enjoy!

Eugene ZaikonnikovPJSUA support in CL-PJSIP

· 74 days ago

After a bit of tweaking, CL-PJSIP now supports basic PJSUA API. PJSUA aggregates much of PJSIP functionality in a handful of structures and protocol methods. This simplifies application side a lot: one can get by with just a few lines of setup code and a couple callbacks. From Lisp perspective it also reduces the used FFI surface to a stable, generic interface. This ought to improve long term compatibility with PJSIP own revisions.

I refer to cl-pjsua-demo.lisp in the library for a short sample. Load the demo system and try (cl-pjsip::run-pjsua "sip:[email protected]") for a quick test against a voice menu directory. It was tested on Linux x86_64 with CCL 11.1 and Allegro 10.1 beta. It however eventually crashes with floating point exception on SBCL.

Zach BeaneErlangen intro

· 76 days ago

Max Rottenkolber made a distributed, async message-passing system. Here’s his introduction to Erlangen.

McCLIMProgress report #4

· 79 days ago

Dear Community,

During this iteration I have continued to work on the tutorial, improving documentation, working on issues and assuring CLIM II specification compatibility.

Most notable change is that argument type in define-command is not evaluated (but if it is quoted it still works for backward compatibility reasons). I've also started some refactoring of the frames module implementation.

The tutorial work takes some time because I try to fix bugs when I encounter them to make the walkthrough as flawless as possible. While I'm not overly satisfied with the tutorial writing progress and its current shape, this work results in benefit of improving the code and the documentation.

The documentation chapter named "Demos and applications" has been updated to reflect the current state of the code base. Some additional clarifications about the pane order and pane names have been added to it. I've updated the website to include the external tutorials and include the Guided Tour in the Resources section. The manual has been updated as well.

The rest of the time was spent on peer review of the contributions, merging pull requests, development discussions, questions on IRC and other maintenance tasks.

Alessandro Serra has created a Raster Image Backend - a backend similar to PostScript, but having png as its output. See "Drawing Tests" in clim-examples and the chapter "Raster Image backend" in the Manual.

A detailed report is available at:

https://common-lisp.net/project/mcclim/static/documents/status-reports.org

If you have any questions, doubts or suggestions - please contact me either by email ([email protected]) or on IRC (my nick is jackdaniel).

Sincerely yours,
Daniel Kochmański

Eugene ZaikonnikovAnnouncing CL-PJSIP

· 83 days ago

I am pleased to announce CL-PJSIP, a Common Lisp wrapper for PJSIP, a popular multimedia communications library.

CL-PJSIP so far supports a limited subset of PJSIP functionality, yet sufficient to implement a simple SIP telephony user agent. Things will get gradually expanded from that. At the moment, focus is on moving beyond alpha-quality (scary amounts of FFI there) and implimenting Lisp-ideomatic handling of PJSIP components.

There is a certain learning curve involved in using PJSIP, and it's worth starting with the included cl-pjsip-ua.lisp. This is a near verbatim copy of PJSIP's own simpleua.c.

The application runs here (disclaimers apply) on CCL 1.11 LinuxX8664, although it does seem to run on recent SBCL (albeit not really tested). There are couple +DARWINs in the code, where it was inferred from PJSIP header files, but it was not tested on MacOS at at all.

Hans Hübner

· 83 days ago

Berlin Lispers Meetup: Tuesday November 29th, 2016, 8.00pm

You are kindly invited to the next "Berlin Lispers Meetup", an informal gathering for anyone interested in Lisp, beer or coffee:

Berlin Lispers Meetup
Tuesday, November 29th, 2016
8pm onwards

in November *new location*

xHain hack+makespace
Gr�nberger Strasse 14

U5 Frankfurter Tor
S Warschauer Str.
Tram M10, M13
Bus 240, N40
(location via OSM)

In case you don't find us,
please contact Christian: 0157 87 05 16 14.

Please join for another evening of parentheses!

LispjobsCommon Lisp developer, m-creations, Mainz, Germany

· 91 days ago

Full time position for German speaking Common Lisp developer near
Frankfurt, Germany.

We are a small German software shop based in Mainz, Germany, founded in
2000. We create custom software solutions for mid-size to big companies
in finance/payment, health care, and media research.

Our portfolio includes high volume ingestion of different social and
classic (print/tv/radio) media streams into NoSQL databases,
dictionary-based natural language processing, fast user-facing search
solutions, and visualisation of results.

To expand the portfolio in the areas of natural language processing and
machine learning, we are looking for talented engineers who ideally have

– 3+ years of software engineering experience

– a solid background in Java/C#/C++

– experience in Common Lisp (not necessarily professional)

– experience in one ore more of the following tools/frameworks: CouchDB,
ElasticSearch, Cassandra, Kafka, Mesos/Marathon, Docker

– experience in development of microservices

– experience or strong interest in different machine learning
methodologies such as neural networks, bayesian networks, support
vector machines etc.

Experience with languages and frameworks is not as important as
curiousity, intelligence and open-mindedness. You will get the necessary
time to learn the missing skills. We are interested in a long-term
relationship rather than just staffing a project with ‘resources’.

Due to our size as a small company, we do care about each one of our
colleagues and react flexibly to the (sometimes changing) necessities of
their life. Together we try to develop a plan for your personal career,
depending on your own goals.
Curious? Please contact Kambiz Darabi at [email protected]
He’ll be happy to give you more information and answer all your
questions.

m-creations gmbh
Acker 2
55116 Mainz



For older items, see the Planet Lisp Archives.


Last updated: 2017-02-16 13:59