Categories
Programming

Language of the Month: Lua, part 2

I’m a few days behind schedule since Lua was June’s language, but here it comes, some summary of my experience, after setting things up in the first part.

A Trip to the Moon / Le Voyage dans la Lune
Lua is not this painful, indeed, just the opposite

Overview

I really liked this language, and I’m sure I’ll keep learning it. Very useful and lots of interesting concepts. I haven’t seen an embedded language before, and some parts

Most of what I picked up was from Beginning Lua Programming and Programming in LuaLua Missions was another very useful experience: a “fill in the blank” type of source code collection, which drilled my understanding of the Lua concepts. Some sections felt a little short, so maybe later I’ll think whether there are any more questions I can come up with. Very well worth the time!

There are two things that stand out: tables and local environments.

Tables

If I have to refer to my knowledge of Python, then Lua tables are like Python lists and dicts rolled into one and then some. Here’s one such table:

x = {1, 2, 3, y = "something", ["or else"]="what"}

The first three elements are giving an array (and one can loop over them), then the last two are key-value pairs. One tricky thing is that the “y” there is actually a string (the same as [“y”]), just made it more readable with syntactic sugar. Since variables (references to variables) can be keys as well, this can get confusing if one has a variable called “y”… There are quite a few things about tables that are confusing, but actually pretty consistent so my fuses are not blowing that much as they were in the beginning (for example: how the length is looked up, how to iterate through the different values) .

They are very complex animals so some things are quite complex to do, which in turn enables quite complex things to be done. Pretty much any kind of data structure can be built upon them. So all in all, I’m enjoying them with all their quirks. Also, “weak tables” are an interesting concept (where certain table entries can garbage collected).

Scopes

In Python, the name spaces and environments seem to be on the sidelines (at least what I’ve seen so far): people need it and have to us it, but nothing more than that. In Lua this is taken to a whole new level: scopes can be essential parts of the way algorithms are implemented and neat tricks can be played.

I tried to come up with some good example and failed, here’s a “duh” one:

function newCounter ()
    local i = 0
    return function ()   -- anonymous function
        i = i + 1
        return i
    end
end

c1 = newCounter()
print(c1())  -- 1
print(c1())  -- 2

This is a bit like Python’s “yield” type of generator, without anything special.

Also, in Python there are only two kinds of variables (I think): local and global: either just for you or for everyone. Here they have one more: upvalues, where the variable is local to a parent, so children environment can access them but the parent’s parents cannot. They are strange beasts as well, e.g. the code itself cannot really know if a variable is upvalue or global, only that it can access that.

And of course these scopes help embedding to not just be possible but straightforward and powerful. Local environments can be restricted, some things selectively enabled, and all this transparently, so the script running might not even know that it’s being in the Matrix.

The Lua Experience

Good

  • Fast, both start up and running
  • Language itself is quite small, possible to wrap one’s head around quite easily
  • Written in ANSI C, so basically on every architecture where one can run C, one can have Lua as well
  • Quite a few program has Lua interpreter or interface built in, see the list of these at Wikipedia
  • Tables are great and once one mastered them, are really Swiss Army Knives
  • Built in interpreter (though could use a little love like iPython)
  • The “…” syntax for variable number of arguments
  • Astrological naming convention – modules are rocks, the exercises I linked previously are missions, web framework is Kepler

Bad

  • There seem to be a lot of reserved names (e.g. in metatables) that would be pretty much impossible to figure out with a book (and/or an iPython-like helpful environment), though this is a problem probably only when starting out with Lua

Ugly

  • While some complex things can be solved, some simple things can require complex coding (e.g. in case of the tables when using “__index” and “__newindex“) and there are things that are not even possible. Or I just misunderstood some sections of the book.
  • Using a lot of syntactic sugar (especially in the object oriented section), that I feel need some more practice to pick up, as things can be done in multiple ways
  • The tables’ syntactic sugar covered way of handling string keys, convenient when things go well, mighty confusing at even the slightest problem
  • Too few projects using Lua, too few sources of information (this is not the language’s fault, though)

Project

Originally I wanted to do some useful programming in every language I learn during the Language of the Month series. I failed with Scala (lazy), and now failed with Lua too (underestimating the difficulty). I’m using VLC a lot and it has Lua scripting enabled, so I wanted to do a plugin that can be useful for me later as well. This didn’t work out very well, mainly because it feels the VLC Lua interface is not very well documented, one mostly have to find some examples that exist already and use them. I ended up wasting a lot of time and still don’t quite understand how to do certain things I would like for my code, and don’t even know if it’s possible to do or not. Will come back to it later, until that here are a few links that helped me to get started:

  • VLC Lua FAQ – detailed list of available API functions, but almost no context and examples
  • Lua extensions in the VLC git repo – one can pick up something by checking out those examples
  • Example: Streaming Radio Player Extension – someone who have gone further down this road than me and has a working extension. Very educational, especially to see how concise and straightforward it can be, because the VLC Lua API seems to be thought out well.

Again, hopefully pick this up later (I mean it).

Categories
Programming

Language of the Month: Lua

New month, new language. So far in this series:

I think I do want to amend the original rules set up for this Language of the Month series. At first I though I can write a new project in any language I learn. That is probably too ambitious. So new rules: every month write a new project in the given language OR contribute to an open-source project in that language. This should take away most of the stress and add some social aspect as well. :)

Lua

I’m intrigued by this language because of it’s niche: scripting language within other software. How one sets out to do something like this? What are the requirements for the language, in terms of design, syntax and so on?

Lua programming language logo
See, the moon!

First impressions

So far I was trying out some code snippets and example scripts. First thing to notice was that Lua is pretty darn quick to start. And pretty quick to run as well, though I haven’t used it for very heavy computation yet. E.g. the “get_all_factors” code at the Lua Crash Course (using a larger example number, 1029384756) is timed at ~7ms, wheres the same version on Python (the language that I probably know best and compare other languages to) runs ~10x slower. The snappiness of a scripting language is a surprisingly happy feeling. :)

Tables look very interesting, how the same thing can implement several data structures at once (it’s like Python’s dict, list and kinda-struct at the same time).

Quite intriguing that the Lua Virtual Machine is stack based, probably that’s the reason the language is so embeddable. A while back I was looking for a language with small resource (especially memory) requirements. I had a few suggestions (Dalvik, Forth) that were interesting, but maybe Lua is the one?

The “local” keyword (and the reasons for having it) seems to be a possible source of many harder to debug scenarios. Got to keep this in mind, again not being in Pythonland anymore.

Interesting how the “function” can be defined inline, so looks like there’s essentially no need for a special “lambda” keyword, at the expense of tying a few characters more…

The interpreter is quite well done (with it’s intelligent indentation probably even better then ipython, though would love to see more colours ^^)

As an exercise, here’s a version of FizzBuzz:

function fizzbuzz(num)
   --[[--
   The usual FizzBuzz from Jeff Atwood
   Doing it a bit roundabout to try some Lua language features
   --]]--
   local answer = ""

   if (num % 5) == 0 then
      answer = "Fizz"
   end
   if (num % 3) == 0 then
      answer = answer .. "Buzz"
   end

   if answer ~= "" then
      print(answer)
   else
      print(num)
   end
end

-- Run the thing
for num=1,100,1 do
   fizzbuzz(num)
end

Links

This section will be updated gradually as I find more information.

(Edit: added the second part of this experiment.)

Install

Tutorials

Books

Articles and info

Source of code

Projects using Lua

Categories
Programming

Language of the Month: Scala, part 3

This is a post I should have written yesterday, actually, since it is already June, ready for the next “Language of the Month“. Still, let’s finish off this May Edition.

Taking score

First, I should have spent more time on practicing Scala, as I haven’t actually finished anything in the end. It is probably an excuse, but after using Python for such a long time, getting used to a compiled language with its own weird path-, naming- and import conventions, was just a little bit too much. Mostly I was reading the Pragmatic Bookshelf: Programming Scala, practicing the examples within. That book uses Scala 2.7.x and the current version is 2.9.0.1, so there are a few things that work differently and I ended up having strange error messages with little to no clue how to fix them (mostly imports, and some method signatures must have changed as well). So it was a limited but exciting success.

Scala screenshot
Using them example code

All in all, I liked the language even if I don’t have a clear usage case for it in my mind (just yet). Some bullet-points of my experience:

Good

  • Different way of thinking about “batch” operations (i.e. foldLeft)
  • Once I figure out the Actors that well that I can write related code without referring to the tutorials, that’s going to be a very powerful tool
  • Pattern matching, pattern matching everything. This is one thing that blown my mind in the little Haskell I checked so far, and missing from Python a lot.
  • Flexibility with class definitions, they can handle so many different situations in a very logical and powerful way

Bad

  • Length of compilation and program start. Will have to get used to it, though I read some hints that this can be improved.

Ugly

  • Pretty frequent need of using things directly from Java, so one is never quite independent
  • Special methods that have reserved name for certain functionality (act, apply, …) that are non-obvious, and not easily distinguished from the arbitrarily defined methods.

Future

Definitely will come back to it and learn it in more detail. It looks like a fine language to do parts of larger projects in. In the meantime I will update the link collection in the previous post with new links that I find. I certainly notice more and more Scala posts on Hacker News these days.

Categories
Programming

Language of the month: Scala, part 2

For the last 3 weeks or so I was checking out Scala, and will do for a little while more, so just want to do a little catch up where things are at the moment.

Start

I tried to find a couple of starting points to start to know about Scala. First thought video tutorials or talks would be useful, so been around checking on YouTube. It seems the mainly referenced intro is Scala: a scalable language. Well, it told me a lot about how Scala compares to other languages and how is it better than those, but not much to start learning it. Though the talk made it sound like it is one of the best and most modern languages out there, taking the most useful ideas from loads of other languages.

Had a copy of the Pragmatic programmer: Programming Scala, and just finished going through it. By that I mean that skimmed all the chapters and did try some of the code samples that looked interesting or strange. Not bad, but from the book I just have a very limited answer to the question of: “but what is it really great for?” Concurrency, sure, but that is probably not all.

I don’t know a language before I code enough in it, but before I know it I don’t have to much inspiration what to code. In this case, programming puzzle sites come very handy, though not all of them support Scala. I went to Coderloop because they do and I can submit my solution for performance testing, or could use Project Euler because they are language agnostic (one sends the results only).

Experience

Since Scala is built upon the Java Virtual Machine and supports pretty much all of Java, it feels a little Frankenstein language: every now and then, even for some things that should be quite commonly used, things have to be programmed in Java. Lots of language features are cool, though, even if most of it feels just syntactic sugar. In a way, maybe the whole language is just that? It’s not bad, makes things very concise and powerful, but always feels there’s something more in the background.

Actors and concurrency are pretty cool, though. In the talks I’ve seen people compared them to Erlang, just even more powerful. This is something more to play around with, would like to make something interesting, scalable and fault-tolerant – those are the fun stuff aren’t they? (But this also makes me want to try Erlang soon so I can compare).

Since I was doing mostly Python lately, it feels a bit weird to use a compiled language. For simple programs, it takes quite a long time to do that compilation, enough to wind me up a little when I was troubleshooting. That said, maybe I’m doing a little bit too much “programming by accident“, so I deserve it? Also, starting a Scala program takes a while as well, though once it is up and running, it’s pretty fast.

At this point, I’m not totally sure yet what project to make until the end of the month. I need more programming itches to scratch. The only ideas I came up so far are self-playing games with actors. Or some server/provider. Well, just keep brainstorming, something better has to come up. Maybe when I dive a little bit more into it, since at the moment I cannot really “think Scala” yet.

There’s also a web framework for it, called Lift, which is probably not surprising. The close connection wit Java (and hence with so many of enterprise software) and event based, fault tolerant concurrency is just too juicy. Not sure if anyone’s hosing it, tough (got to check out), either way I can just try it on my local network. I wonder how does it compare to Django, since that’s what I have a bit of experience with.

Links

Information

Example sources

Companies using Scala

Others writing about Scala

(Last updated: 2011 June 23)

Categories
Programming Taiwan

Taiwanese fuel subsidy and the Global Price API

I might have gone a bit overboard with this. Again. It all started when I was reading how the Taiwanese government is planning to implement a fuel subsidy. It bothered me because:

  1. I’m not a big fan of subsidies, since then people cannot make decisions based on the real cost of things
  2. Subsidy comes out of someone’s pocket anyway, so ultimately everyone does pay for it
  3. Looking at the governing party‘s track record and the nearing elections (<1 year), this is likely to be politically motivated
  4. Fuel is pretty cheap compared to other places already, so it must be already heavily subsidized.

The points 1, 2 and 3 are opinions and generalizations. I had to realize that point 4 I just thrown in there without knowing whether it is really true – I just believed it is. Now here’s a good example for [Citation needed].

XKCD: wikipedia protester
XKCD: Wikipedia protester

Filling in the blanks

Yeah, what do I really know? I lived in Hungary and Britain before, and I remember there fuel was more expensive then in Taiwan. Also, it’s open knowledge that the US is pretty cheap, compared to most of the world, that is… But are there any patterns in the price, and what would I expect Taiwan’s level to be?

I was checking around for a while, fishing for the right keywords for the search (“fuel” / “petrol” / “gas”, depending where one’s from, as a starter). Found a couple of sites but they were mostly looking at price comparisons within a country (like Petrolprices for UK) or within a region (like AMZS for Europe, works weird – no real static link, click the UK flag then “fuel prices” in the menu on the left). After a while, however, I did come across a German organization, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) which in fact had a study about international fuel prices. Go ahead and check the 2009 version, it is very interesting to see all that data and some analysis as well. Historical trends, relative prices, some case studies too. I would have loved to see more analysis, but maybe next time, when I’ll be in the in a position to commission such research. :)

Anyway, looking at the Taiwanese data in from 2009, diesel is slightly subsidized while gasoline is, which means that they are somewhat – but not too much – below and above the US price, respectively. In the ranking, Taiwan is very much on the cheap end.

While looking around a bit more on their site, they have a Data preview for 2010/2011 as well. The more recent is the data the better. Took a look at that too. Now Taiwan is a bit above of the US price with both types of fuel, but still on the cheap end. All their data was in a picture, though, that’s not very handy… So did a little data entry, resulting in this datafile.

I was also thinking, whether the economy affects the prices, and if does then how? Wikipedia first for GDP/capita for all the countries of the world, but they were taking the data from the International Monetary Fund, so let’s go to the source. I checked out their data export tool and there were quite a few more fields to choose from. I went with GDP per capita, Implied purchasing power parity, Value oil exports, Value of oil imports (these last two are good catch:). The output is here. The bottom of the page has a download link, downloaded it into this file without modifications.

Next, had to write some analysis code for the whole thing as well, converting the data into suitable database, fixing some errors in the primary database’s data formats. So in the end I had a simple little script that does:

  1. Clean up some of the names (some Unicode errors originally), and fixes one: I do want to convert the original “Taiwan Province of China” into “Taiwan”…
  2. Fix formatting errors in the IMF  data: they used number formats of “12,345.00” instead of just “12345.00”
  3. Fill missing values with “-1” so it’s easy to filter out later
  4. Rank countries by fuel price, ignore countries that have missing economic data or missing price data
  5. Combine all this and print out on the console

(Scripts and data are shared in this git repo.)

Taking a look

I was looking around for some useful visualizer – something that can handle this much data better then an ordinary static plot. Fortunately, Highcharts JS seems to just the right thing…

The first plot I wanted to reproduce is the one from GIZ’s Data preview. Let’s see how it works out:

Instructions: “red”/”blue” countries are net oil exporter/importer respectively, hover over any of the lines to see which country it is, can click-and-drag zoom into area…

So yeah, Taiwan is down at 49 out of 161 countries, and just few net-importer (blue) countries ranking higher. Even those are mostly poor ones.

Now the second picture, how do fuel prices compare with GDP/capita – which I naively think to have some connection to economical power:

Instructions: note that the GDP scale is logarithmic, hover and zoom are the same as before.

Might be just my eyes, but it seems to me that there are two lines on this plot if one ignores the net exporter (red) countries for a second. From the middle to the right prices are increasing: the wealthier countries can pay more for the fuel. On the other hand, from the middle to the left prices are increasing again: poorer countries cannot really afford it. The cheapest (nominally) are the middle-to-poor, $1000-5000 GDP/person countries.

Bottom line: Taiwan is there at ~$18000/¢100, and if there are indeed these lines, then Taiwan is waaaay below the wealthy country line. Based on the economy, the price should be closer to ¢150. This suggest to me that the original assessment was correct: Taiwanese fuel is cheap.

Global Price API

All of this data-hunting and conversion and plotting should not be this much of a pain. I have a feeling there’s need and desire for open access for such information and that transparency would help people’s decision making – whether those people are in charge or part of the public (and should be “ultimately in charge”). Of course, I’m not the only one to say this, and I’m not even a very good one making this happen – just check out Hans Rosling’s TED talk.

I was thinking, how to build a globally accessible database of consumer prices? Fuel is a good choice because it’s universally needed and there are not too many kinds, one can compare apples to apples. On the other hand, there could be other items as well. Maybe recruit a few volunteers from a big bunch of countries so periodically they add more info to a database. Or fully crowdsource it, maybe even the item categories as well? Then build an interface that it can be easily queried and used by programmers and non-programmers as well. Or is there any such database already? Pitching version: “Archive.org for global price and other public data”. Not that I have a business model for this… I’m sure I’d prefer the same as SimpleGeo and completely open the data itself, but I know there are people who will still find opportunities – or make some.

Any thoughts on this? One selfish thought I have is that this would be lovely so I never ever again have to manually enter all the names of all the countries of the world. :) But I do believe there’s much more to this project.