problem. solution

"and here's where I keep assorted lengths of wire" - HJF

Luapower is a custom selection of C libraries chosen for speed, portability, API stability and a free license. On top of that, a growing collection of ffi bindings and Lua modules, all packaged to minimize the friction in putting together Lua/LuaJIT environment for general-purpose programming.

On this page I will try to make the case for luapower based on its main characteristics.

LOW-TECH

Being easy to use is one of the main goals of luapower. This means leveraging technologies that we already know and use instead of creating new ones. It means providing the lowest-tech method of acheiving any task, and providing a more powerful way only as an alternative.

  • the entire luapower consumption workflow is based on http, zip, and/or git, and optionally, a simple shell script for downloading the whole kitchen sink at once.

  • the module production workflow is based on some naming conventions and git, enhanced by few shell scripts for making it easy to work with multiple repositories.

  • the package database is generated by a single Lua script.

  • luapower.com is a static website generated by a pandoc template and just enough javascript to get info from github and from the package database and present it in various contexts.

PORTABLE

The work tree can be moved around from system to system, and even between different platforms, and just work.

This means including the binary dependencies that the code was actually tested against, instead of relying on the OS ones, and making sure that Lua loads the local ones over the system ones.

It also means putting together all binaries for all platforms in the same tree (no branches). If space is really a problem (android) then deleting other platform directories just before packaging the app is easier than checking out per-platform "binary" branches for each module and thus needing multiple work trees.

Finally the work tree also works as deployment tree, thus there's no deployment step, unless you really want to.

MODULAR

Each Lua module has its own life and death, its own repository, commit history, version history (git tags), issues and forks and pull requests.

This allows luapower to evolve gradually and incrementally, allowing old and obsolete modules to stick around for as long as there are users for them, and yet their presence not bother everyone else.

Additionally, luapower modules from different people can be put together to build an entire app by just copying directories over each other, or by cloning git repositories.

GIT & GITHUB

To me the biggest barrier to getting involved in projects online is the friction that our collaboration tools inflict on us. This goes for anything from correcting a typo on wikipedia to submitting a patch to the Linux kernel. Github is just one way to lower that barrier in the realm of programming.

As for git, the reason for having everything under git's control all the time is basically Alan Kay's idea of symmetric authoring and consumption, which says that you have to be able to edit and share back what you consume on the spot. The combination of a code editor and a VCS is the closest we have today to that idea as far as coding goes (:crying:).

NO DIRECTORIES

This is a hard sell, I admit, but I stand by it. Directories are evil. Not so much because of semantics (hierarchies), but because of the tools we use (file browsers, code editors, IDEs, command line) suck at working with them. You don't have instant search in most of them (sublime text excepted), but you have to "navigate" them instead (this is true for GUIs and it's true for the command line as well). Also, one of the major reasons for directory hierarchies is structuring information for human consumption (learning). In the case of a module distribution, the structure of a package is the same for all packages. Thus the juxtaposition of files from many modules doesn't make things less organized. If you feel uneasy about a long flat list of files, try organizing them into directories, learn the structure of the information, then put them back.

Philosophy aside, having all the modules in the same directory is the way require() works by default, which means there's no need to have a custom require loader to find modules.

The downside is that we have to use git with gloves sometimes.

BINARIES

I should not be forced to compile stuff. Not even in Linux. You should let me play around with the library before I even decide if I want to incorporate it in my project or not.

C SOURCES & BUILD SCRIPTS

I know I'm going against the grain with this one, but I deeply hate C build systems and I think for the most part that they are unnecessary. For me, the success rate for going from make to dll on the first shot has always been pretty pathetic. Building with one-liner gcc scripts is much more transparent, which directly affects my ability and motivation to fix problems (which in turn affects the number of packages built). It compiles much faster too.

To me it boils down to how much indirection there is between problem (in this case the .c and .h files) and solution (the gcc compiler). Sure, you have to remember some gcc switches, but they are the same switches for any library that you will ever build.

It also allows you to apply a set of decisions consistently across the entire distribution, like, if you should statically build libgcc or not.

PACKAGE DATABASE

Package type, version, supported platforms, license, etc., this is information that needs to be aggregated and presented in a way that can be sorted and filtered.

DOCUMENTATION

Make it as easy as possible to produce, and as comfortable as possible to consume. For production, this means plain-text with good support for tables (invaluable for tech docs of any kind), and a markup format independent of any wiki engine, web framework or hosting provider (incidentally, the git-flavored markdown fails on both the independence aspect and table support). Tried pandoc (figured a philosopher / Haskell coder dude should know his thing) which has the best table support I've seen, and you can even write custom writers in Lua (and filters in any language that can read and write json). It also supports a variety of markup formats for input, which could come in handy for aggregating 3rd party docs into the website. Which brings me to the consumption part, which reads: docs should start with a quick-ref table listing "function(args) -> return value ..... concise description" - chances are you'll rarely look beyound that once you get past the initial learning curve.

FREEDOM

My code is in public domain (PD) as I do not support the copyright law. The rest is mostly free as-in non-viral, to avoid the case where one module is imposing licensing restrictions on other modules and on your own applications.

PROBLEMS STILL TO BE SOLVED

Federation

My hope and vision is that in the future we will see less and less of the centralized monopolies we have today, and more and more of the decentralized networks that are now only starting to arise. I think this will be true at the society level as well as in technology.

In this world, there is no github service. Github is nothing more than a piece of open source software that people host on their private VPS, and use it to publish their repos and tweak it the way they like. In this world, two computers can talk to one another directly without hole punching, people don't have to "login with facebook" anymore because the identity problem is a thing of the past, which means that the social components of github will continue to work without the need for a centralized database and lots of staff to maintain it. I know this is a major perspective shift from the client-server world we have today. Still, some steps can be make to make software more free and personal in the current situation.

So what if I have a different idea about what luapower should include or how it does certain things or maybe I just don't like the color theme? What if the project dies? Why tie an idea to its maker? Why should I be stuck to crying to github owners to add git --describe --tags to their API (because I want to display that on my website) and then wait years for that to happen? Why do the github people have to have their priorities and preferences aligned with mine? In this respect, github is no more open than photoshop.

So the idea is to make it easy for other people to host their own package website powered by luapower, with the minimum of fuss and resources. This is why luapower is low-tech, so that it is easy for you to understand and make it yours, if you wish to do so.

Integrating 3rd party documentation

Gotta think about this. Feedback welcome.

Integrating docs with code editors

Chopping doc. pages to feed a code editor for displaying tooltips, autocompletion, and inline doc.

Figuring out dependencies of demos and test units

Currently there's no way to load a script/app (as opposed to a module) and track its require calls without actually running it, which could have side-effects. Thus the dependencies of demos and test units are not listed and must be figured by the run-check-error cycle or by looking at the code (no big deal to look at the top of the file for require calls though). Maybe a Lua parser could help here, not sure.