Physicist, hacker. Enjoys avant-guarde literature probably a bit too much. Open source advocate and contributor, both for software and hardware. Follow these posts on the Fediverse by @gergely@gergely.imreh.net
This is the kind of thing I read so many times on Hacker News: someone manages a “page” on Facebook, only to have it disabled/deleted one day, out of the blue. The difference is that this time it was the page I was managing.
A few years back I took over the page for Inkscape, and updated it every now and then. Quite enjoyed it and even a relatively small following (3774 at the last count I had) people were very active. Then this week once when I logged in and wanted to send out an update – I just couldn’t find the page. Disappeared from the search, from the control panel, everywhere. I could only see the page that was created from its Wikipedia article, and there was the Inkscape “group” which is much smaller.
Draw up some good stuff with Inkscape
I was looking around for a long time for a support contact, and there was practically nothing. It’s a hell of a challenge for coming up with Google search keywords because there’s just too much noise for Facebook + support related terms (that’s a pretty bad sign). In the end I go to this form, that supposed to be used to report problems with a page. Too bad half of it I couldn’t fill out. What is my page’s web address? I don’t know, it was “on Facebook”. What happened to it? Have no idea, I haven’t had any notice about it. Attach a screenshot. I wish I could but did I mention that it disappeared? Anyway, after submission I’ve been told that I’ll receive an email and it is crucial to act on it, otherwise the procedure won’t go forward. That’s been 4 days ago, and no email since.
Contingencies
By now I have given up on getting it back, it’s just does not worth the effort. On the other hand, I do have another page, Ignite Taipei which we manage/organize together with a couple of friends. That would hurt much more if that would disappear, especially because there’s not much presence elsewhere on the net just yet.
Strange, but actually this whole issue bothers me much less than it would have about a year ago, even if I got locked into Facebook even more. Maybe I know now better that shit will always hit the fan, whatever you do, just have to be prepared for it.
Please exit in an orderly fashion
So, how about extending the idea of the World Backup Day to social identities? How can I prepare to lose the least in case my pages or even my profile gets blocked/hacked/deleted?
The personal information is the most valuable, who are my friends and how to reach them. Should write a script that can run somewhere in the background, backing that info up periodically. At least names and emails. Maybe even import them into GMail contacts right away…
Establish links with people outside of Facebook. I already use email instead of Facebook messages whenever I can, now I have to extend that really to everyone.
Build a proper site / blog for the pages I’m involved with and make people aware of it. Always have a point of contact outside of FB.
…. what else did I forget? (leave me a message in the comments if you have more idea)
Why Facebook?
Do I ever fret that Google might block me? That Tumblr would delete my stuff? That Twitter removes my account? Nope, didn’t even really occur to me, they are “not like that”. On the other hand, the power Facebook has over online identities makes people cringe. Why are they different? I feel it’s because they not at all transparent in their decisions and also quite arbitrary. Too many innocent were punished to go unnoticed, and every decision is final. Unless of course one is so big like Ars Technica who had contacts within the HQ.
I was reading an article lately how Facebook want to manage their growing pains algorithmically: more math instead of more people. To me that means that there will be just more arbitrariness and even less transparency…. I wish if instead they would have more support people and start to clean up the mess instead. But of course, if I’d know how to manage Facebook, I would have invented Facebook. :)
I’m not a big fan of subsidies, since then people cannot make decisions based on the real cost of things
Subsidy comes out of someone’s pocket anyway, so ultimately everyone does pay for it
Looking at the governing party‘s track record and the nearing elections (<1 year), this is likely to be politically motivated
Fuel is pretty cheap compared to other places already, so it must be already heavily subsidized.
The points 1, 2 and 3 are opinions and generalizations. I had to realize that point 4 I just thrown in there without knowing whether it is really true – I just believed it is. Now here’s a good example for [Citation needed].
XKCD: Wikipedia protester
Filling in the blanks
Yeah, what do I really know? I lived in Hungary and Britain before, and I remember there fuel was more expensive then in Taiwan. Also, it’s open knowledge that the US is pretty cheap, compared to most of the world, that is… But are there any patterns in the price, and what would I expect Taiwan’s level to be?
I was checking around for a while, fishing for the right keywords for the search (“fuel” / “petrol” / “gas”, depending where one’s from, as a starter). Found a couple of sites but they were mostly looking at price comparisons within a country (like Petrolprices for UK) or within a region (like AMZS for Europe, works weird – no real static link, click the UK flag then “fuel prices” in the menu on the left). After a while, however, I did come across a German organization, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) which in fact had a study about international fuel prices. Go ahead and check the 2009 version, it is very interesting to see all that data and some analysis as well. Historical trends, relative prices, some case studies too. I would have loved to see more analysis, but maybe next time, when I’ll be in the in a position to commission such research. :)
Anyway, looking at the Taiwanese data in from 2009, diesel is slightly subsidized while gasoline is, which means that they are somewhat – but not too much – below and above the US price, respectively. In the ranking, Taiwan is very much on the cheap end.
While looking around a bit more on their site, they have a Data preview for 2010/2011 as well. The more recent is the data the better. Took a look at that too. Now Taiwan is a bit above of the US price with both types of fuel, but still on the cheap end. All their data was in a picture, though, that’s not very handy… So did a little data entry, resulting in this datafile.
I was also thinking, whether the economy affects the prices, and if does then how? Wikipedia first for GDP/capita for all the countries of the world, but they were taking the data from the International Monetary Fund, so let’s go to the source. I checked out their data export tool and there were quite a few more fields to choose from. I went with GDP per capita, Implied purchasing power parity, Value oil exports, Value of oil imports (these last two are good catch:). The output is here. The bottom of the page has a download link, downloaded it into this file without modifications.
Next, had to write some analysis code for the whole thing as well, converting the data into suitable database, fixing some errors in the primary database’s data formats. So in the end I had a simple little script that does:
Clean up some of the names (some Unicode errors originally), and fixes one: I do want to convert the original “Taiwan Province of China” into “Taiwan”…
Fix formatting errors in the IMF data: they used number formats of “12,345.00” instead of just “12345.00”
Fill missing values with “-1” so it’s easy to filter out later
Rank countries by fuel price, ignore countries that have missing economic data or missing price data
I was looking around for some useful visualizer – something that can handle this much data better then an ordinary static plot. Fortunately, Highcharts JS seems to just the right thing…
The first plot I wanted to reproduce is the one from GIZ’s Data preview. Let’s see how it works out:
Instructions: “red”/”blue” countries are net oil exporter/importer respectively, hover over any of the lines to see which country it is, can click-and-drag zoom into area…
So yeah, Taiwan is down at 49 out of 161 countries, and just few net-importer (blue) countries ranking higher. Even those are mostly poor ones.
Now the second picture, how do fuel prices compare with GDP/capita – which I naively think to have some connection to economical power:
Instructions: note that the GDP scale is logarithmic, hover and zoom are the same as before.
Might be just my eyes, but it seems to me that there are two lines on this plot if one ignores the net exporter (red) countries for a second. From the middle to the right prices are increasing: the wealthier countries can pay more for the fuel. On the other hand, from the middle to the left prices are increasing again: poorer countries cannot really afford it. The cheapest (nominally) are the middle-to-poor, $1000-5000 GDP/person countries.
Bottom line: Taiwan is there at ~$18000/¢100, and if there are indeed these lines, then Taiwan is waaaay below the wealthy country line. Based on the economy, the price should be closer to ¢150. This suggest to me that the original assessment was correct: Taiwanese fuel is cheap.
Global Price API
All of this data-hunting and conversion and plotting should not be this much of a pain. I have a feeling there’s need and desire for open access for such information and that transparency would help people’s decision making – whether those people are in charge or part of the public (and should be “ultimately in charge”). Of course, I’m not the only one to say this, and I’m not even a very good one making this happen – just check out Hans Rosling’s TED talk.
I was thinking, how to build a globally accessible database of consumer prices? Fuel is a good choice because it’s universally needed and there are not too many kinds, one can compare apples to apples. On the other hand, there could be other items as well. Maybe recruit a few volunteers from a big bunch of countries so periodically they add more info to a database. Or fully crowdsource it, maybe even the item categories as well? Then build an interface that it can be easily queried and used by programmers and non-programmers as well. Or is there any such database already? Pitching version: “Archive.org for global price and other public data”. Not that I have a business model for this… I’m sure I’d prefer the same as SimpleGeo and completely open the data itself, but I know there are people who will still find opportunities – or make some.
Any thoughts on this? One selfish thought I have is that this would be lovely so I never ever again have to manually enter all the names of all the countries of the world. :) But I do believe there’s much more to this project.
I was thinking what what to write about, but then I realized that is doing this whole blogging thing completely the wrong way. I should be thinking what to do and interesting things to write about will come out of that. It always does.
I was watching the Programming Paradigms lecture series from Stanford. I quite enjoy it, mostly because Jerry Cain seems to be a good lecturer. A quick check confirms that others feel that too. I haven’t finished it yet (24 lectures in to the 27 long series, each ~50 minutes), he talked a lot about C, then Scheme, now about Python. He also mentioned a couple of other languages he suggest for the students to try. That got me thinking: I love languages (human and programming as well) and always looking out to learn more things. Why not do some more exploration in this area – trying out languages that are quite different from each other and see what can I take home from each. I did have similar idea last year, “Language of the Week”, but one week is just too little to get to any practical depth in a language and that idea died. Now, I’m hoping to resurrect it in the form of “Language of the Month”. It is conveniently the beginning of a month so I can just start right now. :) Also, let’s make it round – a 12 language series, one for each month until the same time next year. That should be a challenge. So I made a 12 item list for planning:
Many of these I was planning to check out for a long time and now I have proper motivation. Some of them I got intrigued about while I was checking this humbling list. Note, that “?” are not weird languages, but I keep things open to add more as I get more experience with this (you too can give me recommendation in the comments!:). Also note the absence of some languages, like C and Ruby: those I really should know more, that’s why I won’t do them here, they have enough intrinsic motivation, here I want to try somewhat more off-the-path things.
The simple rule for this series is: every month choose one new language and by the end of the month implement something practical in that language. Something that scratches a real life programming itch. Other than this, all bets are off. See what I can learn.
Now the hard part: which one to start with, since the above list is rather a “set” – ordering is not included. I did roll a dice, but didn’t like the result (like most rolls of dice:), so let’s just choose one: The Language of the Month for May is Scala (because I know absolutely nothing about that, unlike some of the others).
I guess when a hacker learns a language it is different from the way “others” do that. I guess this, because I think I’m a hacker, I’m learning a language and it feels different. I see two main factors coming in the picture:
I’m connecting things in my life, so the things I do usually need some motivation or purpose behind them, without which they are abandoned. The activities I keep up the best are the ones that connect multiple different things.
I want to do things as efficiently as possible. Hack the tools, hack the process to make it better. If there are no tools, make some. If no processes, come up with ideas.
Okay, these are pretty vague expressed like this, now let’s give the example that prompted this post: I’m learning Chinese. Living in Taiwan for two and a half years now, so “high time” doesn’t even start to describe it. Finally I got a tutor, and a good one at that, so twice each week I have a good session of chatting and learning. After each of the sessions we have at least 30 or 40 new words and expressions written down. Those would normally be just forgotten, so I take an effort (about another hour or so after the session) to type them into a Google Document, this very one on the picture:
The Google Document that powers my learning (click to enlarge)
Enter the English expression, the Chinese original and the pronunciation in Bopomofo (which I prefer to Pinyin). This last is possible because of Yahoo Chinese-English Dictionary (one service that is generally better than Google’s own Translate, though I frequently need to use both). The other three columns I’ll came in just recently.
At the moment I’m up to 307 expressions, and that’s just not possible to practice from a spreadsheet like that. I remembered, though, checking out a fellow StartupBus participant, Pamela Fox‘s Google Spreadsheet Flash Cards some time ago. It was fun but followed a different logic than me so couldn’t make complete use of it. But then: why not make a new practice system for myself? This goes back to the precious points: 1) connecting programming and languages – in both I’ll learn something new and they’ll reinforce each other’s motivation, 2) use the exact tools that I need even if I have to make them (because it’s possible to do).
Also after having done Who Said That? (that is currently down due to the AWS fail), I was into guessing games: let’s make a vocabulary guessing practice app that uses the above spreadsheet that I have anyways. What format it should be? Well, for the very first test, to see how to interact with Google Docs and such, just made it as a simple, console-based python app, something like this:
The console app to practice me Chinese vocabulary (click to enlarge)
Each practice round has 30 questions, randomized, 4 possible solutions, with the pronunciation as hint. Simple, though very ugly in the inside at the moment (here’s the repo, I still need to write a ReadMe). It works and now I know what to look out for in terms of implementation (how to log in, how to get and update data, stuff like that).
The other 3 columns in the above spreadsheet are explained as well: they are keeping score such as the total number of times a word/expression is practiced, the number of good answers and the number of current good-answers-in-a-row. These provide a rough-and-ready way to diagnose and manage the learning process – until I come up with better ways. Anyway, right now I get about 50%-70% good which is more than I expected, but still a lot more room for improvement.
A list of improvements to the program that I’m thinking about:
Making it into a site, so I can use my phone on the go to practice. Also, potentially others can use it as well to practice anything based on any spreadsheet with “one side – other side – hint” structure.
I’m pretty sure this whole things could be done in a single Javascript powered page. I don’t know enough Javascript to pull it off yet, but I’ve seen that all the components separately, and that would make a very portable and compact solution.
Need to figure out some easier setup of the spreadsheet if this is to be used by others later. I cannot rely on them understanding what they way I was thinking. Maybe in-app option to add more fields?
I remember reading somewhere that the most effective practice is that I reduce the frequency of checking words that I know well. That’s where the last column comes in: as one has more right answers in a row, one can tast that word/expression fewer times. If there’s a mistake then go back to the original method and test it more until score builds up again. This can potentially be a very complicated algorithm, I got to think of a way that scales well (ie. it is not too bad compared to an ideal method but does not require extensive amount of calculation). Have some ideas, but they need more polish.
After watching Salman Khan’s TED talk this year it grabbed me just how much information is there in one’s actions (they do amazing feedback to teachers on how the students study), how much better you can understand why did people what they did if you have all those diagnostic information (ah, the temptation of Big Data:). To apply this idea of extended diagnostics I could have a logging system instead of keeping (a simple) score. From there the system could get: how much you practice, how are you doing / getting better, what are easier or more difficult words for you, what two words you mistake with one another, suggest things to practice more, suggest words from topics that you know well to extend your knowledge… And more (this was just a little brainstorming). I don’t think I’d have time to extend it like that, and ideas are a dime a dozen, but one never knows…
Adding more modes of learning not just multiple choice, multiplayer learning, more game mechanics (achievements, pins anyone? :)
If there are central datasets instead of self-provided ones, then the system can anticipate what are the difficult parts from other students’ performance before you.
Now, let’s just see how will my Chinese improve during all this hacking, since that’s the main point, isn’t it? :)
Some motivation for learning, loads of Taiwanese food :)
Ps: If you have any language learning tricks, let me know in the comments!
I’ve been doing things very differently since I came back from the Startup Bus. Even before that I was writing up lists of technologies I wanted to try – but all there was to it, lists. Now I know one can indeed create something in a short time, so there’s no excuse to not actually trying all those tech. Here’s a little case study of what I’ve been doing in the last ~two weeks.
I had an idea for a game: I love quotations and reading a lot of Twitter I thought there could be a lot of “clevers” and “funnies” and “insightfuls” in there… I follow a lot of people and it is always interesting to see the different style people talk. So let’s take a bunch of people who others might know, find their tweets that make good quotation and let people guess who that tweet belongs to.
Yeah, that’s the whole thing. And all of it mostly made because 1) I like these kind of puzzles, 2) wanted to learn a bit of web app programming. Going ahead and looking at the result: Here it is, Who Said That? :
playing with Who Said That. Here, I missed.
To summarize (mostly for my own education), here’s what I learned in the one week of development and in the one week since going live:
Behind the scenes
First I had to choose what technology it should all be based on. The two main contenders were Ruby on Rails and Django. I was reading a lot of pros and cons of both. I know that almost everyone I met on the Bus were doing Rails, and seen it myself too. On the other hand my go-to language to do things is Python. So they were on more-or-less equal footing. In the end I went with Django, because it is completely new to me and can have a little “niche” experience (not because not many people use Django, just not many who I know) that might come handy later. Also, there will be lots of other opportunities to try Rails in other projects. :)
I also wanted something hosted. Since Rails is out, Heroku is out too (that’s what everyone seems to be using, for a good reason, though I also didn’t want to spend money on it if I don’t have to). I also considered Google AppEngine which is not strictly Django but at least Python. I used before (and it is great at least on the small scale), so wanted to see if there’s any other hosting company out there. Fortunately there’s a startup that seems to be just the right thing for me: DotCloud. They are still in beta, but I got somehow an invite by talking to them on Twitter. They host Ruby, Python, Rails, Django, databases (MySQL, PostgreSQL, Redis), and more. One push deploy, roll-back, quite good docs, … and I love startups so why not help them too by testing their platform. :) So there it is: Django on DotCloud.
I also wanted to try a few different techs, like “can I use Redis for something in this project”? But that’s just the wrong way. Instead I should always ask: I want to do this or that: what’s the right tech for it? Using PostgreSQL and Django at the moment, though would think MongoDB would be a good fit too, but DotCloud does not have Mongo yet.
Getting interesting data
First I thought it will be quite easy, because people seemed to like the Twitter API. It was everything but… Now I learned to be much more careful of what I say: an API can be good for a lot of things, but to get exactly what I want can often mean a lot of wrangling, trial and error, and tweaking my own thought process to think in their developers’ way.
What did I want? “Quote-quality tweets”, that is: not a reply, not a retweet, not a link. Popularity (by some measure to be determined) is optional, but would be nice.
How did it go this time:
“I want interesting Tweets from all kinds of folks, from everywhere!” (If you already know web development and my naivete start to make you sniggle at this point, I know, I know…) That looked like a perfect job for statuses/public_timeline. But that only pulls a couple of Tweets in.
Then maybe the Streaming API? Well, I cannot have access to all the tweets (and probably I couldn’t handle of that traffic either). At my level (“Spitzer”) I can only have access to a random 1% of tweets. So I wrote a little app that was listening to the Stream and filtering tweets that looked like a good quote. I left that running the whole night. How many good tweets it collected? I think it was about 3 (yeah, single digit). How many of those were from people who anyone else would be interested? 0. Well, this failed again.
[There was some more trial and error that I don’t remember very well…]
Then had a spark: keep it simple. Instead of fishing in the big pond, let’s choose the interesting people first, and import what they are saying. Well, duh! So it can be even simpler: set up a Twitter user for the sole purpose of administration. Whoever that user follows will be in my database. Can also use Twitter lists to organize them and get some categories out (e.g. someone can be in Celebrity, Actors and Comedy in the same time). This also lends itself to easy administration within a Twitter interface, don’t have to roll my own.
So with this, I just had to figure out who to follow. At this point I mostly imported people who I know and I think they write well or interesting things.
WeFollow is a good resource for categories and popular people
Using the Similar people link when I added someone new
So far I have about 70-80 people in the database, giving more than 4000 useful quotes. Some people are better than other. I was surprised how many of them just post loads of links but almost no personal content (and these are the things I like to learn in a project like this).
In the end I had something simple working and I enjoyed using. Now let’s make it ready to open up.
Domain Name
This part I didn’t really have to but wanted to: putting it on it’s own domain. I mean the likes of “somethingsomething.dotcloud.com” is okay, but I wanted to be able to make short links to the game and that format doesn’t really leaves much space e.g. if I make a Tweet with a link…
Been looking for a name for the better half of two days. Lots of searching and getting inspiration from Domai.nr. I learned a couple of things. One idea was “Who Said It?”, but .it’s Italy does not let people with not EU addresses to register. GoDaddy goes around this restriction for an extra $19.99/year (they vouch for you or something) but that was just too much for me and I didn’t want to register with them anyways. I do have an EU permanent residence at the moment, but just didn’t want to risk it. There were a couple of other domain names that are restrictive in this sense, and in the end I went with Austria’s .at, for “saidth.at”. It’s okay, not too expensive (the prices very a lot between countries, before I though there’s a more-or-less common price). One thing I didn’t like about it, that now I’m registered with living in “Taiwan, Province of China”. Say WHAT? (Okay, let’s put politics aside, that’s for some other time).
Making it look something
Well, from the design of the page one thing is easy to tell: I’m not a designer. I know I should make something simple, clean (because that’s what I like to use as well), but not entirely sure how to get around doing it. I was looking for CSS speech bubbles for a long time. It seems that most of the designs use the :before and :after pseudo-element, which is cool, but has its own problems: eg. cannot be manipulated from Javascript – if I wanted to make a speech bubble pointing 4-ways and then deleting 3 once the player guessed, then 1) I don’t know a way to use 4 :before/:after elements, 2) cannot alter them…. Well, I’ll look into this later.
In the end I’m happier with this opacity change, and hiding/unhiding of elements. I’m using jQuery to achieve it, and also for the AJAX calls to get the correct answer. It works well enough until I find a good designer who’s willing to improve on the page for not much in return. Or we’ll see…
My favorite part of this design is the relative robustness: I tried it even in the Kindle 3‘s browser, everything worked very well except for the results’ display by setting .innerHTML… Guess I should look around if there’s another way to do that. I bet there is.
Showing it off
Well, this didn’t work too well so far. I try to put shameless plugs everywhere, but I got altogether about 35 pageviews in the last 10 days…. and I think half of that must be me. :) Hurrah for Google Analytics. Things tried/to try
I like the “tweet this puzzle” button, but that got altogether 1 click. Guess people are spammed with linked content already so it does not stand out
Set up twitter account for the page. Not even spam-bots noticed it. The hidden management bot account on the other hand got a few “fans” already… :)
Set up a blog on Posterous. Got to write better content, so far I have 1 lone through-click. Good thing is that when I post something there, it is shown in their new post directories, so that blog got already much more casual “views” than this blog here. Space to improve
Submitting it later to tech sites? Maybe, when I made I finished implementing a few things that I want. Or when I give up implementing them for the moment.
???
Maintenance
Since launch, I keep filing bugs, feature requests and refactor requests in the GitHub issue page of the site (well, just not to forget it). I did fix a couple of things but didn’t make that many new features as I thought. It might very well be because I’m more a “hacker” (i.e. making something work quick) then a developer (making something work well). I need some attitude change in this respect. Even if this is a “just for fun” project, I do have many ideas to improve it beyond the current stage. It would be too bad to let it die out of laziness.
Future
Keep improving. See whether there’s something the main domain (saidth.at) could be used for. See if I can pivot it to be something more interesting to people. See where did I go wrong with my assumptions that what makes good game mechanics (since I have 0 experience in that field before:). Also, there are other ideas popping up in my head all the time, so got to think what is better: getting some of the other things done or improve this.
If anyone have any comments, feel free, I take this as an experiment and want to learn as much as possible from it. If you are making things for fun as well, tell me in the comments, I’d love to see! :)
