ExternalResources
Pages 12
Clone this wiki locally
Resources on Unix Shells, Programming Language Design, and Implementation
Active Alternative Shell Languages
Fish Shell -- Probably the most popular non-POSIX shell. Focused on the interactive use case, but also has a new programming language. Written in C++.
Oh Shell -- Goal: combine good programming features and an interactive shell. Influenced by Lisp, has prototypical inheritance, CSP with channels and threads.
Oh is a new implementation of the Choc Shell in Go. This master's thesis has a great historical overview of shells and a motivation for putting programming language features in a shell.
NGS (Blog by Ilya Sher)-- Next Generation Unix Shell. Idempotent actions. Interactivity (browser UI). Written in C.
Elvish -- Goal: shell as a real programming language; structured data like lists and maps. Similar syntax to sh. Interactive shell enhancements. Written in Go.
Mash Shell (Github) -- An object shell. Dealing with structured data. Written in Scala.
EShell in Emacs -- A bash-like shell embedded in Emacs. Example Syntax, Mastering EShell.
PowerShell -- Microsoft's shell that deals with structured data, and has "workflows". Not a Unix shell. Depends on a big VM. 850K lines of C# code.
Dormant/Historical
shok (Github) -- Cross-platform shell, appears to be dormant as of 2014. Hacker News Thread. Written in C++ (with Boost), and some Python.
es shell -- "A shell with higher order functions". Based on Lisp and has garbage collection. Influenced by the rc shell.
rc shell -- The Plan 9 Shell. No word splitting. Feature set is very similar to Bourne shell or bash, but the syntax is entirely different. "Nobody really knows what the Bourne shell's grammar is".
Awk/Sed-Like Languages
Tab Language -- An interesting statically-typed, non Turing complete language that apparently fills the niche of Awk. Written in C++.
Miller (Language Reference) -- Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. Written in C.
TXR -- TXR is a pragmatic, convenient tool ready to take on your daily hacking challenges with its dual personality: its whole-document pattern matching and extraction language for scraping information from arbitrary text sources, and its powerful data-processing language to slice through problems like a hot knife through butter. Many tasks can be accomplished with TXR "one liners" directly from your system prompt. There is a TXR Lisp and then it is embedded in a pattern language. Sort of like the reverse of a template language?
jq -- jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text. Written in C.
Dormant
r17 (Github, written in C++) -- A flexible, scalable, relational data mining language. No releases since 2013.
- r17's syntax is a cross between UNIX shell and SQL.
- Built-in concurrency, including cross-machine concurrency.
- Strong type checking at stream-header-read time.
Interactive Shells
This section is for shells which are only interactive -- they do not define a new syntax for programming, i.e. they borrow the syntax from another language or combination of languages.
xonsh -- Xonsh is a Python-ish, BASHwards-looking shell language and command prompt. The language is a superset of Python 3.4+ with additional shell primitives that you are used to from Bash and IPython. This is focused on the interactive use case, and isn't its own programming language. It's written in Python and exposes Python to the user.
Internal Shell DSLs in Various Languages
Every language seems to have an internal DSL for shell commands. This approach is probably OK for small things, but I haven't seen it in the wild in major pieces of software.
plumbum in Python -- The motto of the library is "Never write shell scripts again", and thus it attempts to mimic the shell syntax ("shell combinators") where it makes sense, while keeping it all Pythonic and cross-platform.
sh in Python -- sh is a full-fledged subprocess replacement for [multiple Python versions] that allows you to call any program as if it were a function. I wouldn't call this a shell because it doesn't support pipelines and such, but it's an example of programmers preferring the syntax of their language to the syntax of Unix shell.
pysh -- Dormant project where the author encountered problems in the approach of embedding shell in another language.
Shell in Ruby -- It provides users the ability to execute commands with filters and pipes, like sh/csh by using native facilities of Ruby. This is in the standard library?
psh in Perl -- Perl Shell (psh) combines aspects of bash and other shells with the power of Perl scripting. This one is notable because Perl has a heavy influence from shell, sed, and awk. It appears it's still not close enough.
scsh in Scheme -- Scsh has two main components: a process notation for running programs and setting up pipelines and redirections, and a complete syscall library for low-level access to the operating system. Oil also aims to have a complete syscall library.
inferior-shell in Common Lisp -- This CL library allows you to spawn local or remote processes and shell pipes. It lets me use CL in many cases where I would previously write shell scripts.
Ammonite-Ops in Scala -- a library to make common filesystem operations in Scala as concise and easy-to-use as from the Bash shell
HSH in Haskell -- HSH is designed to let you mix and match shell expressions with Haskell programs.
Caml-Shcaml: an OCaml Library for Unix Shell Programming
Internal Build Tool DSLs
As with shells, each language community has explored idea of using their language to express build rules.
SCons in Python -- Configuration files are Python scripts--use the power of a real programming language to solve build problems
Rake in Ruby -- Rakefiles (rake's version of Makefiles) are completely defined in standard Ruby syntax. The book Beautiful Code has an essay by Matz which discusses why this is possible and nice in Ruby.
Jake in JavaScript -- A Jakefile is just executable JavaScript. You can include whatever JavaScript you want in it.
Grunt in JavaScript -- This is called a "task runner" rather than a build tool. A shell is also a task runner! The Gruntfile can execute arbitrary code and do I/O, i.e. read package.json files.
Shake in Haskell -- Shake is implemented as a Haskell library, and Shake build systems are structured as Haskell programs which make heavy use of the Shake library functions
sbt in Scala -- Scala-based build definition that can use the full flexibility of Scala code
Shell Complements
ShellJs -- This is the opposite of a shell in JavaScript -- it's all the Unix utilities in JavaScript, and you use JS as your shell language.
Scientific Workflow Languages
TODO
POSIX Shells
See list starting at http://www.oilshell.org/cross-ref.html#bash
TODO: add links
GNU bash -- most popular shell in the world, on Linux, Mac, Windows
ksh / pdksh / mksh -- ksh was a proprietary extension of the original Bourne shell; pdksh was an open source clone of ksh; mksh is a fork of pdksh and used on Android.
dash / busybox ash (same lineage)
busybox hush -- shell in one file.
yash -- ground up. Includes line editing library. Debian package.
zsh (not POSIX compatible by default)
Publications and Academic Papers
TODO:
- Thompson shell
- Bourne Shell
- ksh paper (Usenix)
- bash papers
A Pipe Has Two Ends. Using APL in a multiprocess/multiprocessor environment. -- 1989. A proposal for a flexible but easy to use syntax. ... I offer for general consideration a device that allows data to be piped out of APL, through (a series of) shell commands, and back into APL.
TODO
Videos
Programming Design and Implementation
TODO
Books
Programming Design and Implementation
See http://www.oilshell.org/cross-ref.html
TODO
Tools / Software
See http://www.oilshell.org/cross-ref.html
Pages with Collections of Links
Unix Shell on Wikipedia -- Some useful links, most of which are on this page.
List of build automation software on Wikipedia
UNIX Shell Implementations on rosettacode.org
comp.unix.shell thread -- A huge list of links to shell implementations.