@2017-02-16 13:59 · 3 days ago
I’ve seen this idiom a few times when working with a binary stream:
(let ((elements-read (read-sequence buffer stream)))
...)
While the return value is the number of elements read in that situation, it’s a useful coincidence in a common case. The actual return value of read-sequence is:
the index of the first element of sequence that was not updated
Therefore, treating the return value as the number of elements read is completely wrong if you use the start option to read-sequence.
For example, if stream has one or more elements available to read:
(read-sequence buffer stream :start 42 :end 43)
=> 43
The actual number of elements read is 1, not 43.
@2017-02-09 23:56 · 9 days ago

It's been a good while since I last wrote an entry. I didn't write anything all January for a multitude of reasons, at the forefront being that it was exam season again. That's over with now, fortunately, so I do have some more time on my hands to goof off. Unfortunately though, I only have one more week left of this precious "free" time before university strikes me in the back again. I best use it wisely.
The first week of my holidays now I've spent on a heavy reworking of Portacle, a portable, multiplatform, upgradable, install-less Common Lisp development environment project. I've only just now finally got it to work properly on all three platforms and differing versions thereof. It has been an unbelievable struggle, as every platform posed unique problems and nasty edge-cases that were hard to catch without testing everything on a multitude of virtual machines all the time. If you look through the commit history you can find plentyful examples of the borderline insanity that I reached at points. I might write an in-depth article about my adventures another time.
During the exam session itself I was mostly occupied with working on Lichat, my own light-weight chat protocol. It has the nice properties that it is very uniform and rather simple to implement a client for. Among the nice aspects of the protocol are that everything goes over a channel, even private messages, that each update is associated with an ID and failures or responses can be tracked even if they happen to be out of order, and that connection multiplexing is provided directly by the protocol, so you can log into the same user account from multiple machines or instances simultaneously. I've written both a TCP and a WebSockets server, and a CL and JS client for it as well, so you can easily set up your own distributions of it. These CL and JS clients also have respective user interfaces as well, namely LionChat and Lichat-JS. Lionchat might turn into a generic chat client at some point in the future.
I've also continued work on Maiden, and finished a Lichat client for it as well. It's now running as a bot alongside the previous Colleen and I'm intending on switching over fully soon. The only thing that I haven't switched over yet is the chat logging, as I haven't had time to test that extensively yet. I've also been using its Text To Speech module to provide some fun interactivity for my video game streams. Before I can unleash Maiden onto Quicklisp I'll need to heavily document everything though, as it is quite a lot more complex -and capable- than Colleen was.
Then, I've also taken the time to switch over TyNET to the new Radiance 1.0. Radiance is now complete and ready for release as far as I'm aware. I've been holding off on releasing it for good as I've been waiting for some feedback on its tutorial from a couple of friends of mine. Once they get ahead with that I'll type up a good, long entry to announce everything. If my paper for ELS 2017 gets accepted, I will even be presenting it at the symposium in April. Speaking of the symposium- I've been working on a new website for it on and off, to which it will be switching in a while. You can get a preview peek at it here.
For Christmas I gave my father a Raspberry Pi 3 and installed Emacs and CCL onto it. He's been using it, and a DAQ board, to interface with a Korg synthesizer, since he's interested in making music on a more low, analog level. To do this I've been trying to teach Lisp to him since Christmas. While it's been a hard road so far, he's slowly getting into it and making advances. Given that he's been a Fortran programmer for almost all his life, with probably close to forty years of experience or more in it, I'm actually surprised at how well he's been doing with something that probably seems very different and alien to what he's used to. As part of this, though, I've had to write a few libraries, namely cl-gpio to interface with the Pi's GPIO pins, cl-spidev to use the serial port interface, cl-k8055 for the DAQ kit he's been using, and pi-plates for the pi-plates extension boards. The last library isn't done yet as I haven't had the time to test it yet, but the other three should work just fine.
So, I've been slowly, but surely, working off my long stack of projects that need to be completed someday. With Maiden out of the way soon too, I'll finally be able to get back to two of the remaining major projects: Trial, my game engine, and Parasol, my painting application. I'm still undecided as to which I'll tackle first, though. If you have a preference, let me know!
Finally, aside from programming work, I've gotten back into the swing of drawing. Before I was constantly feeling down and unmotivated about it. Everything just seemed to suck and I could never come up with ideas for things that I actually wanted to draw. I'm still not exploding with ideas now, but at least I've found something that inspires me a lot. Specifically, I was reading through Yokohama Shopping Trip and it was filled with exactly the kind of scenery-heavy, romantic drawings that I always wanted to make. "Romantic" referring to the Romanticism art movement. For the past few days then I've been trying to get a scenery drawing done every evening. You can see them on my tumblr. I still have a very long way to go and I'm painfully aware of all the things I do badly at or can't figure out how to draw properly, but I just have to keep on keeping on. It's too late to throw it all to the wind now.
In the near future I'd like to focus some more on writing things again, and perhaps also on actually reading some of the books that I've started accumulating over the years. But then, I'd also like to continue streaming video games, as I've been enjoying that quite a lot. Ah, there's just so much I'd like to do. If only someone were to figure out the age-old problem of finite capacity. Or in the very least: if only I didn't have to worry about earning any money to make a living, I could just merrily focus on being as productive as I can be instead of having to focus on what's marketable. Ah well, you can't have everything, eh?
So, in the more immediate future you can, if you want to, look forward to a full release announcement of Portacle, Radiance, and Maiden, more drawings, and hopefully more articles and stories as well. No promises, though. I'm not very good at keeping to deadlines for projects.
@2017-02-08 14:07 · 11 days ago
Here’s a message from Andrew Shalit to openmcl-devel:
To the CCL Community -
As many of you know, Gary Byers has been on a leave of absence from his Clozure CL work to attend to his health. I'm writing to let you know that, unfortunately, this leave will be permanent. Gary has resigned from his position at Clozure and will no longer be the lead developer of Clozure CL.
Over the last 30+ years Gary has made a tremendous contribution to the Common Lisp programming community, first through his work on Macintosh Common Lisp, and then through the creation of CCL. Gary has always been ready to share his deep knowledge of the Common Lisp standard and the issues attending its implementation. I don't know anyone whose understanding of the details of Common Lisp could match Gary's. His many contributions will be missed.
The work on CCL will continue, though. In Gary's absence, Matthew Emerson will be stepping up as the primary maintainer of CCL. Matthew is moving CCL to GitHub, and he hopes that others will pitch in to continue the development of this great software platform. Clozure Associates will also continue to support CCL, through direct contributions and through our consulting work. I'll be supporting those efforts, and I hope you'll consider supporting them as well.
Yours,
Andrew Shalit
Clozure Associates
@2017-02-06 15:00 · 13 days ago
I'm happy to announce CL-VIDEO, a basic AVI/MJPEG video decoder written in Common Lisp. The library leverages CL-JPEG for frame processing and CL-RIFF for container format handling.
The code has only been lightly tested on SBCL 13.x/Linux x86-64. Some sample files can be found here (the toy plane AVI) and here.
To run it, CL-JPEG version 2.7 is required. Since it's not in Quicklisp yet as of time of writing, it must be cloned into your local-projects. Then you can load cl-video-player system and run (cl-video::play <your filename.avi>).
The implementation is still very much naive, as expected from a weekend project. There is no support for indexing, and a number of edge cases of legit AVI encodings might fail. The library will decode video frames in the order they occur. No parsing of BITMAPINFOHEADER structures; the assumption is they are MJPG DIB. No audio playback, although the stream is being read and adding at least PCM/WAV playback shouldn't be too big an effort.The decoder is factored out into independent implementation in cl-video.lisp. A primitive CLX media player is included in player.lisp. Each AVI instance contains one or more (audio or video) stream record objects, which hold ring buffers of frames with read and write cursors. The interaction between the decoder and player frontend runnning in separate threads is done via mutexed access to this ring buffer. The player thread can be kicked off a callback supplied to decode method: it will be called once the header part is parsed.
Since CL-JPEG doesn't use any SIMD extensions, the performance is modest. On my 6 year old 3GHz i5 (running one core of course) it decodes 480p 25fps file without hiccups, but going beyond that would require implementing multicore support in decode. Still it might be useful as is in applications not requiring real-time decoding performance.
@2017-01-25 13:27 · 25 days ago
New projects:
- ahungry-fleece — A general utility library of convenience functions and features. — GPLv3
- asd-generator — Automatic directory scanner/generator for .asd project files. — GPLv3
- cl-cache-tables — A wrapper around native hash-tables to facilitate in-process caching of common lisp data structures. — MIT
- cl-feedparser — Common Lisp universal feed parser — LLGPL
- cl-gpio — A library for the Linux GPIO kernel module as used on hobby kits such as the Raspberry Pi — Artistic
- cl-k8055 — Bindings to the k8055 DAQ hobby board. — Artistic
- cl-online-learning — Online Machine Learning for Common Lisp — MIT Licence
- cl-spidev — A library for the Linux SPIDEV kernel module as used on hobby kits such as the Raspberry Pi — Artistic
- cl-xdg — freedesktop.org standards handling — GNU General Public License
- cl4l — esoteric CL essentials — MIT
- glisph — Glyph rendering engine using OpenGL shading language — MIT
- lichat-protocol — The independent protocol part of Lichat. — Artistic
- lichat-serverlib — Tools to help build a server using the lichat protocol. — Artistic
- lichat-tcp-client — A simple TCP client implementation for lichat — Artistic
- lichat-tcp-server — A simple TCP server implementation for lichat. — Artistic
- lichat-ws-server — A simple WebSocket server implementation for lichat. — Artistic
- lionchat — A GUI client for the Lichat protocol — Artistic
- lisp-binary — Declare binary formats as structs and then read and write them. — GPLv3
- omer-count — A script to assist in counting the time period between Pesach and Shavuot. — GPL V3
- ook — A CL compiler and enviroment for literate Orangutans. — Public Domain
- opticl-core — A library for representing and processing images — BSD
- rate-monotonic — A periodic thread scheduler inspired by RTEMS. — GPL-v3
- timer-wheel — A timer wheel implementation with BORDEAUX-THREADS backend. — MIT
- translate-client — A client to online web-server translators, currently only google translate — MIT
- trivial-escapes — C-style escape directives for Common Lisp. — Public Domain
Updated projects:
acclimation,
alexa,
anaphora,
architecture.service-provider,
asdf-system-connections,
asdf-viz,
beast,
binfix,
caveman2-widgets,
caveman2-widgets-bootstrap,
cl-ana,
cl-ansi-term,
cl-arxiv-api,
cl-association-rules,
cl-autowrap,
cl-csv,
cl-cuda,
cl-custom-hash-table,
cl-dbi,
cl-digraph,
cl-enumeration,
cl-freetype2,
cl-glfw3,
cl-gobject-introspection,
cl-hamt,
cl-jpeg,
cl-liballegro,
cl-libyaml,
cl-marshal,
cl-opengl,
cl-opsresearch,
cl-permutation,
cl-qrencode,
cl-readline,
cl-routes,
cl-statsd,
cl-syslog,
cl-typesetting,
cl-unification,
cl-wayland,
cl-yaml,
clazy,
clinch,
clml,
clobber,
closer-mop,
clss,
clx,
croatoan,
curry-compose-reader-macros,
deeds,
defenum,
dexador,
esrap,
fare-quasiquote,
fare-scripts,
fare-utils,
fast-io,
femlisp,
fixed,
format-string-builder,
gsll,
inquisitor,
jonathan,
lambda-reader,
let-plus,
lisp-unit,
local-time,
log4cl,
mcclim,
metabang-bind,
mito,
mk-string-metrics,
modularize,
neo4cl,
new-op,
ningle,
oclcl,
opticl,
parachute,
parse-float,
parser.common-rules,
piping,
plump,
postmodern,
postmodernity,
prove,
psychiq,
qtools-ui,
queues,
restas,
retrospectiff,
rfc2388-binary,
serapeum,
spinneret,
static-vectors,
stumpwm,
sxql,
trivia,
trivia.balland2006,
trivial-ldap,
trivial-rfc-1123,
trivial-update,
ubiquitous,
uiop,
unix-opts,
varjo,
vgplot,
weblocks-utils,
xhtmlgen,
zlib.
Removed projects: cl-ledger, weblocks-examples.
To get this update, use: (ql:update-dist "quicklisp")
Enjoy!
@2017-01-18 15:00 · 32 days ago
In course of last few months there were numerous small changes introduced to CL-JPEG. None substantial enough to warrant own announcement, but taken together perhaps it's due for an update. So here we go, as of version 2.6:
- Pre-allocated buffers in
DECODE-IMAGE and DECODE-STREAM are now supported. This should help reduce consing in bulk-processing applications. The buffer can be pre-allocated based on dimensions via JPEG:ALLOCATE-BUFFER.
- CMYK JPEG support. YCCK to CMYK conversion is now performed by decoder. To convert into 3-component RGB, use
JPEG:CONVERT-CMYK-TO-RGB convenience function.
- An option to disable colorspace conversion via
:COLORSPACE-CONVERSION NIL supplied to decode functions has been added. Can be useful e.g. if one needs the luminance component. Support of the corresponding option for encoder to improve performance in transcoding applications is in the plans.
- Small bugfixes and performance tweaks.
@2017-01-18 11:54 · 32 days ago
@2017-01-18 11:43 · 32 days ago
@2017-01-02 15:52 · 48 days ago
They’re looking for a “full stack Java programmer,” but must also have 3+ years hand-on experience in Lisp/Tk Programming on Linux.
See web technologist job posting.
@2016-12-31 15:33 · 50 days ago
Today (December 31) is the last day for the
Quicklisp appreciation fundraiser. It has met and exceeded the matching fund goal by a wide margin. I am so thankful of all the support for this fundraiser, but also for the many kind words of appreciation, and for the "Quicklisp supporter club" donors, over the years. The positive feedback has made the work very satisfying.
Thank you, and best wishes for a happy 2017!
@2016-12-28 00:00 · 53 days ago
Dear Community,
During this iteration I have continued working on the tutorial, fixing
the issues and assuring CLIM II specification compatibility.
The most notable change is standard-application-frame implementation
refactor which simplifies the code and moves some computation to
creation time (until now we have searched through sheet hierarchy at
each iteration of command loop and at various other occasions at
runtime).
The rest of the time was spent on peer review of the contributions,
merging pull requests, development discussions, questions on IRC and
other maintenance tasks.
Alessandro Serra has created a Framebuffer Backend - a working proof
of concept that McCLIM may work on top of a frame buffer. Albeit a bit
slow you may find instructions how to run it here:
https://github.com/robert-strandh/McCLIM/wiki/CLX-backend(s)
A detailed report is available at:
https://common-lisp.net/project/mcclim/static/documents/status-reports.org
If you have any questions, doubts or suggestions - please contact me
either by email ([email protected]) or on IRC (my nick is
jackdaniel).
Happy new year!
Sincerely yours,
Daniel Kochmański
@2016-12-27 18:32 · 53 days ago
This season finds us remembering Wilde's remark about fashion being so unbearable that it must be utterly destroyed every six months. In that spirit, we submit to current fashion in documenting how ABCL repl may be obtained locally viz:
docker run --tty easye/abcl /usr/local/bin/abcl
Happy holidays!
One may need to explicitly update the base openjdk8 container that easye/abcl is based on via:
docker pull easye/openjdk8
docker run easye/abcl /usr/local/bin/abcl
[Docker Engine]:
https://www.docker.com/products/docker-engineIf you want a specific packaging to add your own Java/Lisp artifacts, you may easily rebuild the Docker image that the container starts with from cloning
the ABCL source and then issuing:
docker pull easye/openjdk8
docker build -t easye/abcl .
See
http://abcl.org/svn/trunk/abcl/Dockerfile for build instructions.
@2016-12-27 13:01 · 54 days ago
Orangutans grok Common Lisp
This being the season to be jolly (and eat bananas), I produced a magical piece of software: a compiler and decompiler for the language
Ook! Which proves that orangutans grok Common Lisp.
You can try out this recreational and magical piece of software by downloading the library from
common-lisp.net. A brief introduction can be found, of course, in the
library (if you can see it!)
Please send bananas and DO NOT say "monkey"!
Enjoy
(cheers)
@2016-12-22 14:43 · 59 days ago
When I tried to find Nikodemus Siivola’s nice post about load-time-value and inline caches, I found his site, random-state.net, was no longer working.
At my request, he managed to get it up and running at a new address long enough for me to make a mirror of all his posts and articles.
Until the “real” location is fully restored, you can access the content of the site at random-state.xach.com, including the nice article about load-time-value and inline caches.
When random-state.net comes back, I’ll redirect everything back there.
Enjoy!
@2016-12-21 11:32 · 60 days ago
Make FFI callabacks work properly on 64 bit Windows OSes. � sharplispers/cormanlisp@2156983:
Making a project open source is no guarantee that anyone will care. But in the case of Corman Lisp, Artem Boldarev's hard work on fixing problems has not only revived the project, but revived Roger Corman's interest in it:
Artem, thanks so much for this fix!!!!! I believe this was the one the caused me to stop working on it for some time, and ultimately just open source it. I wasn't sure anyone else would be able (or willing) to track it down. I had dealt with and fixed many such issues during corman lisp development. However when I hit this issue specific to 64-bit OS (and those were fairly rare then, but of course now ubiquitous) I spent some evenings and just decided I couldn't do it any more. I got as far as noticing that the callback code broke and was getting a garbled address (I think an 8 byte address/4 byte address mismatch), and this probably happened when an access violation occurs (these are employed for garbage collection heap management). Now that I see it is working well on 64-bit systems I am inspired to get back into it.
Thank you, Roger, for sharing the source, and thank you, Artem, for diligently hacking on it!
@2016-12-20 03:22 · 61 days ago
@2016-12-19 14:05 · 62 days ago
tamamu/glisph:
I like the demo GIF!
@2016-12-19 00:00 · 62 days ago
Announcement
Dear Community,
After almost a year of development we are proud to present a new
release of ECL tagged with version 16.1.3. All changes are backward
compatible fixing bugs and other issues, implementing new interfaces
and cleaning up the code base.
ECL manual has been updated in a few places. Work on a new
documentation is still pending. New documentation is still incomplete,
but you may see it here:
https://common-lisp.net/project/ecl/static/ecldoc/.
Before this release we have performed extensive tests on many
platforms (Linux, FreeBSD, OpenBSD, NetBSD, OSX, Windows MSVC, Windows
MinGW, Windows Cygwin, Android and Haiku). For details please consult
https://gitlab.com/embeddable-common-lisp/ecl/issues/307. Extra
attention has been paid to Windows testing to improve that platform
support.
This release is available for download in form of a source archive:
Best regards,
ECL Development Team
API changes
Added better interface for package-locks.
Introduced functions:
ext:package-locked-p package
ext:lock-package package
ext:unlock-package package
ext:without-package-locks &body body
ext:with-unlocked-packages (&rest packages) &body body
To use these functions user has to require the module
(require '#:package-locks)
defpackage accepts new option lock to allow locking package on
creation:
(defpackage foo (:lock t))
mp:holding-lock-p: introduce new function for multiprocessing. Function
verifies if lock is hold by the thread which calls the function. Usage:
(mp:holding-lock-p my-lock).
make-random-state: fix problem with simple-vectors. The correct
initialization types for make-random-state are: (OR RANDOM-STATE
FIXNUM (MEMBER T NIL)).
Initializing a random state with an appropriate array (element type and
arity dependent on platform) is also possible.
ext:random-state-array: new extension for random-states. Usage:
(ext:random-state-array random-state).
ext:terminate-process: new extension for external processes. Usage:
(ext:terminate-process process) with a second, optional boolean
argument whenever termination should be forced or not.
Enhancements
Implemented CDR-7.
https://common-lisp.net/project/cdr/document/7/index.html
implemented CDRs: CDR-1, CDR-5, CDR-14. Both CDR-1 and CDR-5
were already implemented, CDR-14 made us to list them in *features*
(https://common-lisp.net/project/cdr/document/14/index.html).
if ECL is build with --with-cxx option, :CXX-CORE is present in
*features*.
deprecated configure option --with-local-gmp has been removed - use
--enable-gmp (defaults to auto).
configure options has been revised.
ASDF has been upgraded to version 3.1.7.26 (with a few patches scheduled
for 3.2.0).
bundled CLX has been purged. Lately I've fixed ECL support on portable
CLX maintained by sharplispers on https://github.com/sharplispers/clx
(available via QuickLisp).
initial port for the Haiku platform. The port is done by Kacper Kasper's
work, one of Haiku developers. Threads are not supported yet.
refactored ECL internal tests framework. Tests in src/tests are now
asdf-loadable (with load-source-op) and divided into test suites. make
check target runs all regression and feature tests which aren't supposed
to fail.
removed 15000 lines of obsolete code. Files not included in the
buildsystem but lingering in the codebase or options failing to
build. All info is added in the new documentation in the section "Removed
interfaces".
improved man page and help output. Man page now contains up-to-date list
of flags, as well as explanation of flag's behavior.
deprecated long flags with one dash, added two-dash version. Flags that
aren't one-character, but start with one dash (e.g. -eval) are now
deprecated; long version --eval was added instead.
indented C/C++ code to follow emacs's gnu C style. This is a first step
towards coding standards in the documentation. Additionally all in the
src/c/ directory are listed in the appropraite documentation section
(new-doc).
refactored list_current_directory in unixfsys.d. Function was
obfuscated with ifdefs with non-even pairs of #\{ and #\}.
Issues fixed
ECL signals floating point exceptions in top-level console.
mp:rwlock is treated as built-in class (previously process crashed if
class-of was called on such object).
ECL builds now succesfully with --with-ieee-fp=no option.
ext:file-stream-fd: doesn't cause an internal-error if called
with something not being a file-stream (signals a
SIMPLE-TYPE-ERROR condtition).
stable-sort: bugfix and improvement in speed. Adapted from SBCL by
Diogo Franco.
typep: accept * type specifier as abbreviation of T as described in
2.4.3 Type Specifiers of the specification.
MOP: fix problemes when redefining non-standard and anonymous
classes. Bugs identified and fixed by Pascal Costanza.
getcwd: fix issue with too long pathname. This fixes the regression,
which crashed ECL at start when pathname exceeded 128 characters limit.
make-random-state: fix a problem with simple-vectors. Until now #$
reader macro accepted simple vectors as an argument, what lead to bugs if
vector didn't match specific requirements like the element type or the
arity. Now we sanitize this.
make-load-form: provide implementation for random-state objects.
thread fix on msvc: on windows importing thread was closing the thread
handler so the thread wakeup wasn't working because the handler is not
more valid.
import thread wasn't set upping a proper environment: on some case the
thread was mistakenly thinking that the thread was already registered.
ECL_HANDLER_CASE and ECL_RESTART_CASE didn't work as expected. Bug
identified and fixed by Vadim Penzin.
@2016-12-16 13:27 · 65 days ago
@2016-12-16 12:31 · 65 days ago
In my real work, I (or better, my students and colleagues) use
R to program our data analyses (if you are interested, you can check out the
TRONCO package.)
Of course, being a Common Lisp quite-high priest at this point and a
recreational programmer in it, I wondered how to render several
R constructs in the language with parentheses. It turns out that it was quite fun to re-engineer some of the key features of the
R language.
This is not the first such project trying to mimic
R. I know at least two of them; the most public one being
Common Lisp Stat, but, being known for suffering from NIH and RTW syndromes, I had to jump in.
Here is a short summary of what I did and what was achieved.
Generic Math, Arrays and Vectors
R provides generic math on vectors. Pretty much everything in
R is a vector (stored in
column major order; more on this later) including arrays. The generic math helps you writing things like
> 42 * 1:5 # The ':' operator produces a vector.
[1] 42 84 126 168 210
Indexing
The generic math and the construction of some operators in CL is pretty much straightforward. Where things get hairy is in the handling of the several ways that a
R programmer may use to index, slice and dice vectors and arrays. As an example, you could write
> a = array(11:14, dim = c(2, 2), dimnames = list(list("a", "b"), list("x", "y")))
> a
x y
a 11 13
b 12 14
In brief, you have a matrix with "named" rows and columns. There are other idiosyncrasies but this is already something interesting to deal with, as you can now write things like:
> a[1, "y"]
[1] 3
> a["b", ]
x y
12 14
The second "indexing" selects, or
slices, the entire second row (counting from 1) and, as a result, returns a vector
with the column names retained.
Indexing can also become interesting because of other idiosyncrasies in
R. Consider the following:
> a[2]
[1] 12
> a[42]
[1] NA # 'NA' is the 'not available' indicator.
> a[c(2, 3, 4, 10)]
[1] 12 13 14 NA
Indexing by only one index accesses the underlying vector in column major order. If the index is beyond the current limit, the value
NA ('not available') is returned. If the sole index is a vector, then the indexing returns a vector of the values in the array accessed column major order.
Indexing and Assignments
Of course, once you handle slicing and dicing of arrays, you have to deal with
setting these slices and dice.
In R you can have a lot of what you can dream of (and something more, some of which, you wished was not there).
Basic assignment on vectors and arrays works as expected. Consider:
> a = array(11:19, dim = c(3, 3), dimnames = list(list("a", "b", "c"), list("x", "y", "z")))
> a
x y z
a 11 14 17
b 12 15 18
c 13 16 19
> a[2, "y"] = 42
> a
x y z
a 11 14 17
b 12 42 18
c 13 16 19
So far, nothing special is happening. Things actually make sense when trying to set a sub-slice of the array.
> b = array(42, dim = c(2, 2))
> b
[,1] [,2]
[1,] 42 42
[2,] 42 42
> a[2:3, 2:3] = b
> a
x y z
a 11 14 17
b 12 42 42
c 13 42 42
However, when setting a sub-slice of an array to a vector, or setting the elements of an array using a single vector index (i.e., when accessing the array in column major order), things may become a bit surprising.
> a[2:3, 2:3] = c(1, 1, 1, 1)
> a # This is not a surprise.
x y z
a 11 14 17
b 12 1 1
c 13 1 1
> a[2:3, 2:3] = c(1, 2, 3, 4, 5, 6) # This is also expected (modulo the 'multiple'.)
Error in a[2:3, 2:3] = c(1, 2, 3, 4, 5, 6) :
number of items to replace is not a multiple of replacement length
> a[c(1, 2, 6, 7)] = c(111, 111, 111, 111) # Ok, understandable.
> a
x y z
a 111 14 111
b 111 1 1
c 13 111 1
> a[c(1, 2, 12, 7)] = c(1, 1, 1, 1)
> a # What?!?
[1] 1 1 13 14 1 111 1 1 1 NA NA 1
Ok, the last example shows how
R manages assigning past the limit of the
underlying vector.
Not really a problem, once you get the hang of it, but, source of a bit of confusion nevertheless, at least to yours truly.
In any case, all of this must be dealt with once you want to get into building some facilities like these on top of Common Lisp.
Let's "Reinvent the Wheel" (RTW) in Common Lisp
Getting the basics down was not so difficult. Common Lisp has a somewhat richer set of array and vector data types than
R, however, some design decisions resulted in some fun (albeit probably sub-optimal) setup.
Here is a list of decisions taken, which may be subject to debate.
- Objects are represented as structures.
- Constructors follow the R conventions (no
make-array!)
- Vectors and arrays are mapped to CL vectors and arrays.
This means that
- Arrays are kept as multi-dimensional entities which are accessed in row major order
- Vector and array "objects" are separate and different.
- A generic reference function
AT is introduced (and its counterpart, (SETF AT).)
Of course, these choices will make porting code from R to the CL library/tool quite cumbersome; let's say that this goal was dropped early on. The CL library (which, by the way, is named ρ, that is:
RHO) will look familiar to a
R programmer, but it will not be the same. After all, is it Common Lisp, isn't it?
Here is a run down of the basic features. This was the hard part, in the opinion of the author. Getting more of
R implemented will be somewhat easier.
The AT Generic Function
The AT generic function works as expected on the standard CL objects (as it should).
RHO 10 > (defvar qwerty "qwerty")
QWERTY
RHO 11 > (setf (at qwerty 3) #\R)
#\R
RHO 12 > qwerty
"qweRty"
RHO 13 > (at qwerty 4)
#\t
RHO 14 > (defvar a #2A((1 2 3) (4 5 6) (7 8 9)))
A
;;; ...
RHO 16 > (at a 1 1)
5
RHO 17 > (setf (at a 0 0) 42)
42
RHO 18 > a
#2A((42 2 3) (4 5 6) (7 8 9))
The generic function
AT also works on structures and class instances.
RHO 22 > (defstruct foo a s d)
FOO
RHO 23 > (defvar foo (make-foo :s 42))
FOO
RHO 24 > (at foo 'foo-s) ; In this case the accessor function FOO-S is invoked.
42
RHO 25 > (setf (at foo 'foo-d) "Food")
"Food"
RHO 26 > foo
#S(FOO :A NIL :S 42 :D "Food")
So far, so good. Now let's have some fun.
Creating Arrays (and Vectors and Matrices)
Let's create some arrays, vectors and matrices.
RHO 28 > (setf rr (range 1 9)) ; This would be 'rr = 1:8' in R.
1 2 3 4 5 6 7 8
RHO 29 > (setf rv (vector 1 22/23 -3.2 42 5.8))
1 22/23 -3.2 42 5.8
RHO 30 > (setf rv2 (vector* (list 1 22/23 -3.2 42 5.8)
:names '(a b c d e)))
A B C D E
1 22/23 -3.2 42 5.8
The objects returned by
VECTOR and
VECTOR* (the full constructor) are wrappers around the underlying CL vectors. Their class is
RHO:VECTOR. Note also that
*PRINT-READABLY* is set to
NIL, therefore, a "pretty printing" of the vectors is produced.
RHO 37 > (setf ra (array (range 1 9) :dim '(3 3)))
[, 0] [, 1] [, 2]
[0, ] 1 2 3
[1, ] 4 5 6
[2, ] 7 8 1
RHO 38 > (at ra 1 1)
5
The function
RANGE excludes the upper limit (following CL conventions), while the
ARRAY constructors "recycles" the values passed to it (following
R conventions). The value produces is pretty-printed, while its class is
RHO:ARRAY. Indexing is 0-based, as in CL.
Creating matrices is straightforward and, again, a
R-like function is used.
RHO 3 > (setf rm (matrix '(1 1 2 0) :nrow 2 :ncol 2))
[, 0] [, 1]
[0, ] 1 1
[1, ] 2 0
RHO 4 > (at rm 0 1)
1
RHO 5 > (at rm 1 0)
2
Slicing and Dicing in RHO
Now that we have all the bases covered, we can proceed to some "slicing and dicing". All the
R indexing and assignment methods are provided in
RHO, modulo the Common Lisp background choices, bugs and unimplemented features; not many of the last kind.
First of all, let's set some
dimension names...
RHO 7 > (setf (dimension-names ra) '(("a" "b") ("x" "y")))
(("a" "b") ("x" "y"))
RHO 8 > ra
[, x] [, y] [, 2]
[a, ] 1 2 3
[b, ] 4 5 6
[2, ] 7 8 1
RHO 9 > (at ra 1 "y")
5
Next we create a new array (a matrix, but in
R, matrices are separate subclasses of arrays)
rb and use it as a value to set a sub-matrix of
ra.
RHO 13 > (setf rb (array 42 :dim '(2 2)))
[, 0] [, 1]
[0, ] 42 42
[1, ] 42 42
RHO 29 > (setf (at ra (vector 1 2) (vector 1 2)) rb)
#2A((42 42) (42 42))
RHO 15 > ra
[, x] [, y] [, 2]
[a, ] 1 2 3
[b, ] 4 42 42
[2, ] 7 42 42
As you can see, everything works as expected.
Remember that we have built a matrix
rm; let's use it to set some elements of the array
ra, according to the R "access-by-matrix" semantics.
RHO 16 > rm
[, 0] [, 1]
[0, ] 1 1
[1, ] 2 0
RHO 24 > (type-of rm)
MATRIX
RHO 27 > (setf (at ra rm) 1024)
1024
RHO 28 > ra
[, x] [, y] [, 2]
[a, ] 1 2 3
[b, ] 4 1024 42
[2, ] 1024 42 42
RHO 29 > (setf (at ra #(1 2) #(1 2)) #(1 1 1 1))
#(1 1 1 1)
RHO 30 > ra
[, x] [, y] [, 2]
[a, ] 1 2 3
[b, ] 4 1 1
[2, ] 1024 1 1
Note however, that, although "correct", no attempt to signal an error is made for incongruous dimensions.
RHO 42 > (setf (at ra #(1 2) #(1 2)) #(11 12 13 14 15))
#(11 12 13 14 15)
RHO 43 > ra
[, x] [, y] [, 2]
[a, ] 1 2 3
[b, ] 4 11 12
[2, ] 1024 13 14
Getting the indexing of different combinations of scalars, vectors, and arrays was an interesting exercise. The implementation is probably bloated and not easily understandable; most probably because of the design choice of keeping the underlying CL vectors and arrays "as-they-are", especially multidimensional arrays.
Missing Things...
Many...
Most important one is "negative" indexing, which does not work completely yet.
Of course, I am limiting this comment to arrays and vectors: other
R data structures, such as
factors and
data.frames are in the making, but all the heavy lifting is now done.
Want to Play?
Well, for the time being, drop me a line (you know where to find me). We can talk about setting up a proper project and discuss some design choices.
Cheers
@2016-12-15 19:35 · 65 days ago
@2016-12-09 17:01 · 72 days ago
The Common Lisp Foundation Quicklisp fundraiser hit its initial goal of $6,000 after just a few hours. There have been many donors and a few very generous donations put it over the top. I am so surprised and grateful that it happend so quickly.
By exceeding $6,000, contributions are no longer doubled with matching funds. But the fundraiser is still going to continue to December 31st as scheduled, and everything raised until then is important and appreciated.
As I write this, contributions are near $9,700. With the matching funds, it’s nearly $15,700. How high can it go?
Thank you for supporting Quicklisp!
@2016-12-08 17:28 · 72 days ago
The Common Lisp Foundation’s Quicklisp appreciation fundraiser is now up and running. I made a brief video to appeal for your support. Here’s the transcript:
Hi, I’m Zach Beane and I’m asking for your support for Quicklisp.
I created Quicklisp in 2010. It’s easy to install and provides nearly
fifteen hundred Common Lisp community libraries with just a few
commands.
I keep things updated every month, and work with the community to get
new projects added and make sure existing projects keep working. I
check builds, I look for problems, and I’ve filed hundreds of bug
reports to keep those problems from affecting Quicklisp users.
The vast majority of this work has been done on a volunteer basis, in
my free time.
But today, the Common Lisp Foundation has started a appreciation
fundraiser for Quicklisp, running until December thirty-first.
I hope to use the proceeds from the fundraiser for a few months of
dedicated work on Quicklisp. In particular, I want to address the main
issues that have kept it in beta all this time, starting with
security and documentation.
If Quicklisp has made your life easier - whether it’s to get started
as a new Common Lisp user, or to easily get all the dependencies you
need in a large commercial project, or if it just helps you hack on a
hobby or homework project - please consider visiting
quicklisp.org/donate for a
contribution.
Thank you!
@2016-12-08 17:10 · 73 days ago
Note: The
Quicklisp fundraiser is up and running. If you appreciate Quicklisp, please contribute if you can.
New projects:- cl-digraph — Simple directed graphs for Common Lisp. — MIT/X11
- cl-directed-graph — Directed graph data structure — MIT
- format-string-builder — A DSL wrapping cl:format's syntax with something more lispy. — MIT
- hunchentools — Hunchentoot utility library — MIT
- l-system — L-system or Lindenmayer system on lists — GPLv3+
- parser.common-rules — Provides common parsing rules that are useful in many grammars. — MIT
- parser.ini — Provides parsing of Ini expressions. — LLGPLv3
- postmodernity — Utility library for the Common Lisp Postmodern library — MIT
- stl — Load triangle data from binary stereolithography (STL) files. — ISC
- whofields — HTML field rendering and input validation utilities written in Common Lisp — MIT
Updated projects:
3bmd,
alexa,
alexandria,
architecture.builder-protocol,
architecture.hooks,
architecture.service-provider,
asdf-dependency-grovel,
beast,
carrier,
caveman2-widgets,
cells,
cepl,
cepl.drm-gbm,
cepl.sdl2,
cepl.skitter,
circular-streams,
cl+ssl,
cl-ana,
cl-autowrap,
cl-bootstrap,
cl-ca,
cl-change-case,
cl-dbi,
cl-dot,
cl-drm,
cl-gists,
cl-growl,
cl-jpeg,
cl-kanren,
cl-l10n,
cl-libyaml,
cl-mediawiki,
cl-mpg123,
cl-opengl,
cl-out123,
cl-pango,
cl-portaudio,
cl-pslib,
cl-quickcheck,
cl-sdl2,
clack,
clavier,
clfswm,
clinch,
cltcl,
coleslaw,
collectors,
croatoan,
dbus,
dendrite,
djula,
documentation-utils,
dyna,
eazy-gnuplot,
esrap,
fare-scripts,
formlets,
gbbopen,
gendl,
geneva,
gsll,
gtk-cffi,
http-body,
hu.dwim.computed-class,
hu.dwim.def,
hu.dwim.presentation,
hu.dwim.serializer,
hu.dwim.syntax-sugar,
hyperluminal-mem,
inquisitor,
js,
kenzo,
lack,
lass,
legit,
mcclim,
metabang-bind,
metering,
mgl,
mito,
modularize,
modularize-interfaces,
neo4cl,
nibbles,
ningle,
oclcl,
opticl,
pgloader,
png-read,
postmodern,
protobuf,
psychiq,
qlot,
qt-libs,
qtools,
quickutil,
quri,
ratify,
retrospectiff,
rutils,
serapeum,
skitter,
spinneret,
staple,
stumpwm,
sxql,
trivia,
trivial-documentation,
trivial-features,
trivial-nntp,
trivial-rfc-1123,
trivial-yenc,
ubiquitous,
ufo,
utilities.print-tree,
utility-arguments,
utils-kt,
varjo,
websocket-driver,
what3words,
wookie,
workout-timer,
xml.location.
Removed projects: cl-xspf, date-calc, elephant, html-entities, lambda-gtk, perfpiece, quicksearch, usocket-udp.
To get this update, use (ql:update-dist "quicklisp"). Enjoy!
@2016-12-07 15:00 · 74 days ago
After a bit of tweaking, CL-PJSIP now supports basic PJSUA API. PJSUA aggregates much of PJSIP functionality in a handful of structures and protocol methods. This simplifies application side a lot: one can get by with just a few lines of setup code and a couple callbacks. From Lisp perspective it also reduces the used FFI surface to a stable, generic interface. This ought to improve long term compatibility with PJSIP own revisions.
I refer to cl-pjsua-demo.lisp in the library for a short sample. Load the demo system and try (cl-pjsip::run-pjsua "sip:[email protected]") for a quick test against a voice menu directory. It was tested on Linux x86_64 with CCL 11.1 and Allegro 10.1 beta. It however eventually crashes with floating point exception on SBCL.
@2016-12-04 22:13 · 76 days ago
Max Rottenkolber made a distributed, async message-passing system. Here’s his introduction to Erlangen.
@2016-12-02 00:00 · 79 days ago
Dear Community,
During this iteration I have continued to work on the tutorial,
improving documentation, working on issues and assuring CLIM II
specification compatibility.
Most notable change is that argument type in define-command is not
evaluated (but if it is quoted it still works for backward
compatibility reasons). I've also started some refactoring of the
frames module implementation.
The tutorial work takes some time because I try to fix bugs when I
encounter them to make the walkthrough as flawless as possible. While
I'm not overly satisfied with the tutorial writing progress and its
current shape, this work results in benefit of improving the code and
the documentation.
The documentation chapter named "Demos and applications" has been
updated to reflect the current state of the code base. Some additional
clarifications about the pane order and pane names have been added to
it. I've updated the website to include the external tutorials and
include the Guided Tour in the Resources section. The manual has
been updated as well.
The rest of the time was spent on peer review of the contributions,
merging pull requests, development discussions, questions on IRC and
other maintenance tasks.
Alessandro Serra has created a Raster Image Backend - a backend
similar to PostScript, but having png as its output. See "Drawing
Tests" in clim-examples and the chapter "Raster Image backend" in
the Manual.
A detailed report is available at:
https://common-lisp.net/project/mcclim/static/documents/status-reports.org
If you have any questions, doubts or suggestions - please contact me
either by email ([email protected]) or on IRC (my nick is
jackdaniel).
Sincerely yours,
Daniel Kochmański
@2016-11-28 15:00 · 83 days ago
I am pleased to announce CL-PJSIP, a Common Lisp wrapper for PJSIP, a popular multimedia communications library.
CL-PJSIP so far supports a limited subset of PJSIP functionality, yet sufficient to implement a simple SIP telephony user agent. Things will get gradually expanded from that. At the moment, focus is on moving beyond alpha-quality (scary amounts of FFI there) and implimenting Lisp-ideomatic handling of PJSIP components.
There is a certain learning curve involved in using PJSIP, and it's worth starting with the included cl-pjsip-ua.lisp. This is a near verbatim copy of PJSIP's own simpleua.c.
The application runs here (disclaimers apply) on CCL 1.11 LinuxX8664, although it does seem to run on recent SBCL (albeit not really tested). There are couple +DARWINs in the code, where it was inferred from PJSIP header files, but it was not tested on MacOS at at all.
@2016-11-28 09:23 · 83 days ago
You are kindly invited to the next "Berlin Lispers Meetup", an informal gathering for anyone interested in Lisp, beer or coffee:
Berlin Lispers Meetup
Tuesday, November 29th, 2016
8pm onwards
in November *new location*
xHain hack+makespace
Gr�nberger Strasse 14
U5 Frankfurter Tor
S Warschauer Str.
Tram M10, M13
Bus 240, N40
(location via OSM)
In case you don't find us,
please contact Christian: 0157 87 05 16 14.
Please join for another evening of parentheses!
@2016-11-20 14:06 · 91 days ago
Full time position for German speaking Common Lisp developer near
Frankfurt, Germany.
We are a small German software shop based in Mainz, Germany, founded in
2000. We create custom software solutions for mid-size to big companies
in finance/payment, health care, and media research.
Our portfolio includes high volume ingestion of different social and
classic (print/tv/radio) media streams into NoSQL databases,
dictionary-based natural language processing, fast user-facing search
solutions, and visualisation of results.
To expand the portfolio in the areas of natural language processing and
machine learning, we are looking for talented engineers who ideally have
– 3+ years of software engineering experience
– a solid background in Java/C#/C++
– experience in Common Lisp (not necessarily professional)
– experience in one ore more of the following tools/frameworks: CouchDB,
ElasticSearch, Cassandra, Kafka, Mesos/Marathon, Docker
– experience in development of microservices
– experience or strong interest in different machine learning
methodologies such as neural networks, bayesian networks, support
vector machines etc.
Experience with languages and frameworks is not as important as
curiousity, intelligence and open-mindedness. You will get the necessary
time to learn the missing skills. We are interested in a long-term
relationship rather than just staffing a project with ‘resources’.
Due to our size as a small company, we do care about each one of our
colleagues and react flexibly to the (sometimes changing) necessities of
their life. Together we try to develop a plan for your personal career,
depending on your own goals.
Curious? Please contact Kambiz Darabi at [email protected]
He’ll be happy to give you more information and answer all your
questions.
m-creations gmbh
Acker 2
55116 Mainz