Category: Computers

Categories
Admin Lab Programming

Laboratory 2.0 – a monitoring system

Looks like that one of my specialty as a physicist, and contribution to the labs where I have worked so far, is bringing different kinds of programming techniques, and technologies to the table. I’m not saying I’m any better than many of the professors, post-docs, and students I’ve met so far (there are plenty of ingenious ones), it’s more like I experiment with different tools, have tried more of the cutting edge or recent technologies, did some web programming and could whip up something quick – that might not work very well at first, but does broaden the horizon for the rest of the people.

Also, I’m a lazy person, so want to automate as much as possible. That was on my mind recently when we have been preparing to do a vacuum-system bake-out. It’s essentially a procedure to have a delicate experimental system, mostly made up of steel, glass, and stuff like that, closed up from the atmosphere, all the air pumped out, then heated up to high temperature (~150-300°C). One has to be careful, because things can break, there are temperature limitations for some materials, also on how quickly that temperature can change, requiring careful monitoring of the status of the system. And the whole thing takes something like two weeks or more. Perfect setting for automation.

Set up the electronics

The pressure measurements are done by some expensive other equipment so didn’t have to bother with that one yet, so set to work first on the temperature monitoring. Before it was a bunch of thermocouples and multimeters, requiring manual intervention and lots of labour. Instead, got some inspiration from Adafruit’s Thermocouple Breakout Board, using the MAX31855 chip, and also from the Thermocouple Multiplexer Shield. It can handle only one channel, but can use some other chip together with it to switch between the different thermocouples, and so we can read it out one-by-one. The Adafruit board could only handle 1 channel, and the multiplexer shield was using an older chip for the measurement that I could not buy anymore. In the end, found a good analog multiplexer that one that is sold in the computer market here in Taipei, the CD4067B, and it works pretty well.

Breadboard setup for temperature monitoring Arduino
Breadboard setup for temperature monitoring with Arduino

Of course, setting it all up was quite a bit of fun times, as there were way too many gotchas along the way.

  • MAX31855 is a surface-mount component, and haven’t worked with it before. Not too bad, and can be much neater, just takes some plactice
  • MAX31855 is a 3.3V circuit, so the CMOS voltage levels used by my Arduino Mega ADK had to be level shifted
  • Unlike the older chip, MAX31855 really needs differential input, and it’s much more sensitive to the environment. This required different kind of analog multiplexer than that board had
  • The Arduino Mega is a new model for me, and had some strange behaviour in terms of the serial communication
  • Surprisingly there are not too many options for 3.3V voltage regulators over here, just the LM1117, which is different from what others are using elsewhere
  • Lots of noise and stability issues until figured out what should be how. For example under no circumstance should touch the thermocouple to conducting surfaces, and avoid ground loops
  • While MAX31855 says it’s “cold-point compensated”, meaning that it accounts for the chip-s local temperature when measuring the thermocouple, it doesn’t appear completely compensated, meaning that we can have unexpected measurement change because the chip is heating up for example by being in a closed box.
  • Figuring out the right amount of time to wait between switching channels (375ms seems to be good enough, 500ms is totally fine)
In the end, though, we did have a nice 16 channel thermocouple multiplexer, sending off the measurements onto an LCD screen and to the computer over an USB cable.
Temperature monitoring board soldered
Temperature monitoring board in it’s lab setting with 16 thermocouple channels

This is then saved in a database, and can be accessed from elsewhere.

Visualize!

The thing that my co-workers were most amazed by wasn’t the electronics. Sure, they haven’t worked with Arduinos, but did do similar stuff. Instead they liked the monitoring interface much more, this is the one on the picture right here (can click to enlarge)

Bakeout Monitor interface showing the vacuum system, temperatures, pressures and long term graphs
Bakeout Monitor interface (click image for full view)

It’s the schematic layout of our equipment, with the temperatures positioned where the actual sensors are. Also, the change of the measured values in time are also displayed with live scrolling.

I’m not saying it’s great. Thinking about it, the major insight that made it good for the rest of the people is that I realized how much more people understand visual data: the placement of the values to the corresponding locations on the schematics. That’s the only thing.

So inside it’s a MongoDB database (learned from previous mistakes, using a replica-set at least), with Python scripts talking to the sensors and saving the data, NodeJS / Smoothie Charts for visualization (and plain old CSS positioning of <input> tags for the reading display), nginx‘s upstream module for running two monitoring servers just in case. It’s mostly in the Github repo of the monitoring code, as well as the Arduino sketch for talking to the electronics.

It was actually quite fun to write it all, and the gradual improvements, trying the new tech, trying not to lose to much data, amazed how well it works. Especially had a good time learning about the database, scaling, fault tolerance, performance…

Of course there could be room for a lot more improvements.

  • My failover-restart bash scripts are awful, though they do seem to work more or less and counteract the USB unreliablilities
  • There were some changes to Smoothie Charts that I could improve on: logarithmic plotting, some display enhancements, wonder if it can be more optimized for performance
  • More efficient data loading. 12h data is about 30Mb in JSON format, that I send compressed, apparently it gets down to ~5% in size, but it still takes quite a bit of time to process on the frontend
  • The layout now can be changed from config files if the sensors change, so co-workers can do that without programming knowledge. I wonder if that can be simplified even more

Of course, I’m a person who generally overengineers stuff, so maybe it’s good to stop somewhere. And the somewhere might be when I got to the point to use my Kindle for monitoring (craps out on 1h data already, but some real time things are good enough).

Bakeout Monitor interface running on Kindle
Bakeout Monitor on running on Kindle 3, not perfect but does work

Get on with it

I did learn a lot along the way, and I’m sure that with this experience I will be let to do a little bit more in the lab in terms of programming ideas. I don’t like that the rest of the system is currently forced to be LabView, but that’s for another post, and there are so many things that can be improved in general as well. Let’s just go and do that.

Categories
Computers Startups

A different smart dressing up

I was out with a few friends the other day, they were forming a team to go to StartupWeekend Taiwan Hardware next month. I have been to one or two previous StartupWeeekends and they are good fun. Haven’t made up my mind about the hardware one, but since they asked me could they prepare for it, since it’s their first experience, I did try to gather some advice. Not sure how better they are off with it, but I hope at least a little. Looking at the previous events, a little bit of experience and knowledge can put people way ahead, because Taiwan is just learning startups, every experience is golden.

Later, though I started to think what I have told them. One particular advice I had to examine: don’t start with a tech that you find interesting, start with a story instead and choose your tech for that.

Now I’m not that sure that this is a good advice for every occasion, especially because I was brainstorming about one particular tech that really excites me, and if I was right earlier, then I’m wrong now. And the more I thought about it, the more I felt, that there can be a case (especially for StartupWeekends) for starting with a tech, even if it is probably the harder way to get something awesome out of the process. Still, given that limitations make one more creative, it worth doing at least some proper brainstorming about it.

Lilypad Arduino

The tech I was thinking about that time was the Lilypad Arduino. I haven’t got one (it’s sitting in my Sparkfun basket, ready to be ordered), but really want to make a project using it.

Lilypad Arduino Ebroidery
Lilypad Arduino Ebroidery by Bekathwia (click picture for Flickr link)

The Lilypad Arduino is a microcontroller (a very tiny computer) that you can stitch (using conductive thread),  and people most often use it for wearable computing / smart clothes. It has a lot of periferials, could use it with LEDs, accelerometers, buzzers, buttons, light sensors, temperature sensors, wireless communication, vibration motors, and whatnot… Really the imagination is the limit.

So now the question is, what kind of projects I could come up with that could use this. In the last few days of brainstorming (and now writing this), I came up with a couple of opportunities, not sure if any of them has been done. Not sure about the quality either.

Project ideas

Smart bag: figure out how to make some simple/small way of communicating with a little sensor that can be attached to items that one does want to always bring along: keys, wallet, phone…. The bag would sense it if they are out of its range and warn the user. Never leave stuff behind at home or at a cafe.

Visual turn-by-turn navigation for cycles: use a smartphone to get navigation directions to where you want to go with your bicycle or motorcycle. A jacket is outfitted with lights on both arm, and would have a Lilypad to communicate with the smartphone, signaling which way to turn and when.

Movement direction clothes:  smart clothes that would detect the position and posture of the person wearing them, and use light or vibration to signal what movement is the next one. Could maybe correct choreography or teach karate katas.

Communicating clothes: this could be done in multiple ways, wireless, infrared (like the TV remote control, just two way), consciously controlled or in the background. Ultimate spy clothes, send messages between people in the crowd without them actually doing anything. Could local business send out signals that are received the clothes could prompt you with a deal or order ahead your favorite if you are a regular (though, this can really easily be very spammy). Also, the different units can synchronize with each other – cue super visual flashmob.

Smart bedding: what if your bed could monitor your sleep and  wake you up when the morning comes? Bit like Wakemate, but for the bed.

People tracking: build into work clothes for a factory and can log in and out people just by sensing them. Make it part of the ticket in an amusement park, and can see which rides are too popular, can communicate back to advise about waiting time and suggest other ways to pass the time. Finding people in a crowded place in emergency, see if anyone’s left behind. Though these can be very 1984 if done badly….

Health sensing: monitor vital body functions for people who are somehow at risk: older people, partying in town (drink responsively), doing sports…. Warn them when some critical situation is predicted.

Style guide: clothes having smart tags with “style” and “colour” and “pattern”, and either all of them collectively, or a central piece of clothing would check whether these thing you wear do match with each other. Could also make recommendations what to wear. “I have this trousers, shoes, and so on – which shirt should I wear to that?” – press a button, the right choice lights up or vibrates in the closet.

Well, that’s for now. Most of these, I realize, are quite a bit tentative, many of them seem to miss some key ingredient, or have a (technical) problem to solve before it would work. Which one would I bring to a StartupWeekend if I’d go, which one is ready to be made and could be made in 54 hours? Tough one…

Found anything interesting here, or have some more ideas? I’d love to hear them, let me know in the comments!

Categories
Programming

Let’s talk about humidity

It is very much summer now here in Taiwan, and if one thing the Tropics is good at, then it’s being hot and humid. That’s sometimes fun, when there’s enough chance to hang out by a pool or on the seaside, but most of the time not that much. That’s why there are so many air conditioners, and other clima controls.

As I really like data, and this place is great for electronics, I thought I can just combine the two things, and try to learn more about my environment. Temperature is relatively easy, but for a long time I was looking for a humidity sensor. At the local electronics market I found a few different models the other day, and got one of them – pretty much the cheapest one, because all the manuals were in Chinese anyway (which I don’t yet read), and they were also all packed up, so there was not much peeking anyway. There was a combined temperature/humidity sensor board (two sensors already packed up together), but at 450NT it felt pricey and needed to buy some connectors for it as well. There was a larger round capacitive sensor, but it was too large for my liking.

I ended up with a “CM-R Resistive Humidity Sensor” (here’s an English spec scheet for it). It was 250NT (~5 quid or 8 bucks) which makes it relatively expensive equipment for me, though maybe because I was saving my cash in my wallet for going to Mobile Monday Taipei later that night. Still, if it works then it does worth it, and might get a few more pieces.

Wiring it up

I never actually knew how the electronic humidity sensors work, so this is a good primer for that.  Nevertheless, the plan was afoot to hook it up to an Arduino and measure some RH% (relative humidity). It was quite tricky for me to figure out the wiring, though, because I never worked with resistive element that could handle only AC voltage, and gets destroyed by DC.

That white rectangle with black spots, the humidity sensor at work, all plugged in with Arduino

Fortunately there were a couple of people (indeed just 2 or 3 that I could find) who were working with this, and had some good information. The one I went with was this post from the Arduino forums. Was quite informative, though I think the final formula is faulty – not because I understand it completely (my electronics theory is spotty at best) but because experimentally I figured out what works instead (being an experimental physicist, that’s pretty much what I do all day anyway).

Finally it is hooked up something like what is shown in this hideous schematics (really got to find some better way to draw this):

Humidity sensor schematics, done with Fritzing

After this wiring, I have to switch the digital output (DO) pin between High and Low at about 1kHz with 50% duty cycle. When the pin is on High value, every now and then I measure the voltage with the analog input pin. From the measured voltage I can deduce the resistance of the humidity sensor as

RH = R1 / ((Vref - Vm) / (Vin - Vm) - 1)

where Vref is the reference voltage (5V of the digital pin), Vin is the measured voltage, Vm is in this case Vref/2=2.5V, the voltage that the capacitors are charged to during all this switching, and R1 is the reference resistance. It works very well, I tried to measure some fixed resistances, and the results are pretty accurate.

When we have the resistance value, the spec sheet has a look-up table for translating the resistance into relative humidity (RH%). It’s quite cumbersome, because will have to interpolate between values, and it’s also not that accurate if the temperature is not known. On the other hand, the values from 25℃ can be used pretty well for all normal temperature ranges here in the summer (let’s say air conditioned 21℃ to outside summery 33℃) with maybe ±5% error in the humidity measurement, that’s not that bad.

The sensor itself feels really sensitive, just having it in the lab, could sense changes in RH as others were doing water-related things the next room. Even if reading is not that accurate, it is still quite precise, and for most of the things it is still okay. I could try in different environments, like our “dry box”, which maintains ~20% humidity, but that means quite high resistance of the sensor, MOhm instead of tens of kOhm, so the voltage reading just moving around near 5V, making it not that precise and noisy as well. For 30-90% humidity it works pretty well, though.

To monitor the readings, I wrote up a quick web-app from some reused code with NodeJS, Express, Socket.io, node-serialport2 and Flot. There’s no kill like overkill, but at least I can already monitor it from any other computer as well, it’s real-time, and the GUI is much easier to adapt than most other ways I know.

All the code for the Arduino and the monitoring server is in my Weatherstation repo on Github. It’s pretty buggy and not documented, will clean that up later when I’ll have a good working version.

Deployment

Now let’s get down to some real action. Being at home, I often feel that my flat is way too humid. Firing this circuit up, I got a reading of 80-85% relative humidity, that’s not something to sneeze at. Then turned on the air conditioning in dehumidifier mode, and watched what happened.

First there was an increase of the relative humidity, because the drop in temperature is quicker and the drop in water content. After chilling for a while, this is how it looked like.

Humidity logging while running the AC’s built in dehumidifier (RH% vs time in minutes)

Looks like some funky periodic behaviour, but empirically I felt much better, even if the humidity didn’t drop much. Maybe the extra comfort comes from the temperature drop as well, so will try to do the same thing in the usual temperature-control-only mode as well, see what does that one look like? I haven’t had neither my laptop battery charger, nor a spare battery with me, so I couldn’t run this too long, I wonder what level it would reach over a longer time. Looks like it has a downward slope, but it’s hard to tell.

Also, the wobbles are likely because of some feedback control that is turning the dehumidifier inside the AC on and off, and that controller has a huge hysteresis. It is not totally surprising, though: as much as I know, dehumidifiers are just big cold surfaces and the water condenses on them from the passing air. For that you have to keep a large surface pretty cold, and it is likely some time to cool a surface or let it warm back up again, thus the controller likely overshoots in both the de- and the re-humidifying parts of the process.

From what I have tried, it feels like a pretty good sensitive sensor, and coupled with

Plans

I’ve seen another way to wire up the sensor in a tutorial, which is basically a resonant circuit and the chance of the sensor’s resistance will change the output frequency. That frequency can be readily measured with Arduino already, so if I get the few extra components, might give it a try. It seems that potentially it can have better noise characteristics then the method I’m using.

Want to add some real logging function to it, maybe saving readings and timestamps into an SQLite database, which I can analyze properly later with Numpy. Or maybe store it in MongoDB, on which I can still interface and do some concurrent analytics even when the sensor is running. Or maybe I consider that just because I use Mongo for Friendcare?

Would like to add a 10-LED light bar, which could display the humidity decades without any computer, thus making the whole setup so much more portable. Though for that I will definitely have to revive a 74HC595 STP16C596 Serial-in/Parallel-out shift register, because then I’ll need one resistor and a single DO channel to control those LEDs, instead of 10 DO channels and 10 resistors.

Could also add an LCD screen like this SparkFun SerLCD. The only problem is that it interferes with the serial connection, so might not be able to use it together with the computer monitoring. Might worth it, though, that LCD I have (white-on-black with backlight) looks pretty darn funky.

Could also add some long-term monitoring and some other sensors, to build up a real home weather station. I have my eyes on a barometric pressure sensor, but that’s quite expensive.

Update (2012/11/6): 

  • Been using this in our laboratory for almost two months now, it works quite well, though still thinking of there’s any better way to measure the resistance of the sensor. Brushing up on RC circuits is in order?
  • Looks like I’m not the only one thinking about humidity sensors and data logging: there’s also another post from the awesome Tom Igoe.
Categories
Programming

Frindcare, some improvements

After getting to a working level with Friendcare, my web app to track friending and defriending on Facebook, I was still doing some tweaks. Some of the tweaks is to improve functionality, some of them to fix some broken behaviour by the services I use (especially Facebook). This is a little summary of what I have learned in this short time.

Changes

Heroku

The very day of my last blogpost, when I actually had others sign up as well, Heroku had a few hour long outage. That was quite unfortunate, and it was annoying to see the “Application Error” page, which pretty much hides what really happened. In real production environment probably I would have to run things on multiple independent platforms so if the platform itself is the one that crashes, the site could be just switched over. This time I’m just taking it simple, make a personalized maintenance page, which I can turn on in this case, or when I’ll really need some maintenance time. Also, played around Google Web Fonts, so can have the good old school Press Start 2P on this one.

Friendcare is offline for maintenance
Friendcare maintenance screen

It would be nice if in case of trouble I could use a web interface to trigger maintenance mode. Might build a service for that, though now that I think of it, that will have to be hosted outside of Heroku. Also, later might do an application error page as well, just in case. Might even choose a mascot for that. Though probably not.

Fonts

Besides the error page, I was playing around with other fonts for the main interface as well. In the end the current one is Crimson Text, which should be quite readable even with very different font size. It is mostly good, though it doesn’t play completely well with Twitter Bootstrap, the button texts for example are not totally well aligned vertically. Might have to look around the fonts a little bit more.

Friendcare header layout
Tweaked layout, and it actually already has results

Facebook

Facebook has so many weird things going on, that I can barely wrap my head around it. One of my impression, which probably shouldn’t surprise me, that they are most likely not using their own APIs, otherwise it wouldn’t have such huge bugs in it. Another impression is, that somehow they manage make everything almost good, but in some way completely bad.

Graph API reliability

Since the entire site relies on regularly polling the user’s friend list, getting that list consistently is a must. Somehow the Graph API occasionally misses some friends: they don’t appear in the list and I mistakenly assume as lost connections. An hour later, when polling again they are back in the list, so they show up as friendship gain. That doesn’t work very well. This happens a lot, so I had some algorithms in place for that and sometimes had to manually clean up the database (which I really shouldn’t do if it can be avoided).

FQL

I can also use the Facebook Query Language to get the information I want. It is quite straightforward, they even have the expression among their examples:

SELECT uid2 FROM friend WHERE uid1=me()

This is pretty fast as well, maybe half the time of the Graph API request. There’s one problem, though: it returns wrong results. Not entirely wrong, just wrong enough. The list I receive seem to have a bunch of invalid user IDs, such that it doesn’t belong to anyone. They just don’t exists, but I still receive them. So while Graph API suffered from false negatives, FQL suffers from false positives. Fortunately there seems to be a workaround, in the form of a deeper query such that

SELECT name, uid FROM user WHERE uid in (SELECT uid2 FROM friend WHERE uid1=me())

Here I will actually throw away the ‘name’ part, but at least I know that the IDs that I receive do belong to actual people. So far this seems to be the most reliable, though it takes as much time as the Graph API request, or maybe a bit more.

In the end, currently I query both the Graph and FQL in sequence, compare the results, note if they differ (for debugging purposes) and use the FQL result since that seems to be more reliable. Will check back later to this part, when I have more information to go on.

Offline access

Offline access is pretty important for this kind of service, because it’s not that useful if we can only check the results when the user comes to the page. Unfortunately, the ‘offline_access’ permission has just been removed. Instead the access token lives for 60 days, after which one has to get a new one by logging the user in again. I’m not saying it’s unreasonable, maybe even better since I don’t have to ask for any extended permission over the ones that are granted to every app automatically, but have to keep in mind. It might complicate things. Also reminds me that other apps that I’m using with Facebook (e.g. ifttt) will have some problem because of this thus I will have problem with this. Better check with them.

Send message prompt dialog

I kinda understood when they have removed the ability to send message to others on the user’s behalf without their interaction, that’s such a tempting spam delivery system for most, I presume. On the other hand, for a long time at least they had (or I seem to remember they had) a send dialog, where I could prompt a user to send a message to someone else: they write the actual message and they click send. On the other hand, the current Send Dialog has a required parameter ‘link’, this it is no longer possible to send “just a message”. This must be a relatively new chance, since the dialog’s page describes it as:

The Send Dialog lets people to send content to specific friends.

Fair enough, that’s exactly how does it work. On the other hand, just one level up, the Dialogs Overview says this:

The Send Dialog allows a user to send a Facebook Message to one or more of their friends.

Content vs. message, subtle but crucial difference. Thus in my app I cannot have a “send message to this user” button, unless I attach some kind of link to the message. Now this feels really spammy to me.

Overall

It is good to add a few more features, because I can use it better as well, and I learned a lot too. Might add a proper dashboard, a deregister option, notification to email or (if I can figure out how) Facebook message, stats, better layout. Or whatever suggestion I receive.

Finally, the most important thing that it works. In the last week I found two people who defriended me for whatever reason, and I wouldn’t have known about otherwise. They were not close ones, so not going to pursue them, but it could have been otherwise, so at least it’s good to know.

Categories
Programming

Friendcare, satisfying a curiosity

Quite often I find services I’m using inadequate. Something is missing, or not working the way I hope, I wish it could be changed. In a fraction of the cases it can actually be changed, and an even smaller fraction I go and change it myself.

This time it was bugging me that while I am always aware if I have new contact established on Facebook (or colloquially “friended someone”), but there’s just no way to know if someone cut ties (“defriended”) me. In some ways that knowledge is just as important, since the first means I succeeded in being interesting to another person in some way, the second means I was actually annoyance – not just being ignored and never talked to but people took the effort to never hear about me again. Recently quite a few people did that to me, or at least it’s only now that I try to find them for this or that but I can’t. I never want to be caught off-guard like this again, so I made Friendcare.

Friendcare interfacing

It was mostly an exercise in web app building, since if it was only for me, I could have just set up a script on my server that monitors things. It’s more fun as well. First I started it a long time ago, tried to work on it and rewrote from scratch a couple of times until getting here.

What does it do?

In this one the site just gets the user’s friendlist (a list of IDs) from Facebook and compares it with the last known friendlist. If there are new IDs, they are considered friend gains, if there are missing ones, those are friend losses.

Actually it got pretty complicated logic (for my usual thinking) inside because of the asynchronous management of good NodeJS apps, had to think a lot about what information is available at what step and what time of the usage. It also didn’t help that I haven’t really planed the user experience (UX) first, just started to code. Some of the things are surely pretty terribly done because of premature optimization and scaling I was doing while only havng a single user of the site. All in all, it’s an absolutely horrendous mess in the inside that still seems to implement a suitably close approximation of what I wanted. Surly will need some refractoring sooner rather than later.

The parts

It’s amazing how many different parts I need to use to make one complete site work, a total web-tech-soup. Everything is in a library or in an external service. It’s good and bad – good because other people did the heavy lifting for me already, bad because every one of them needs a non-trivial effort to learn and all of them different. Nevertheless, trying to keep things minimal. At the moment it’s NodeJS on Heroku, with MongoDB and using Heroku Scheduler to trigger the update service. The other modules and parts are:

Everyauth

Without Everyauth nothing would work, it handles the Facebook authentication on the backend. Also, it gave me ideas how the site can be extended to multiple services / multiple social networks.

Fbgraph

There are so many interfaces to talk to Facebook, and the heroku-nodejs app I started off from uses one of them as well. That just seems to be too specialized for an example app, not in functionality but in implementation. Instead I found myself another library, fbgraph, and so far so good. If I really wanted to, I could even easier interface just the Graph API, but no need to do that at the moment.

Mongoose

I choose Mongoose/MongoDB to store the associated data, well, basically because MongoLab had the largest free database offering of all the databases I have checked. So far so good. I like the schemaless database, though already run into problems when I was updating wrong fields in the dataset I was surprised that the results didn’t show up at the right place. I guess that’s a danger I can live with at the moment. Will have to learn more about it, how to back things up, how to maintain it well, and about mongoose and it’s schemas.

Underscore

The core functionality is basically taking the difference of two sets, and in general there is quite a bit of set manipulation, so Underscore works very well, I used for a couple of other parts as well (reduce, groupBy), and it works well. The whole thing feels like it should be part of the base library.

Twitter Bootstrap

While risking that I’ll look exactly the same as many others, Bootstrap does give some good-looking results with moderate effort. Will have to do some tweaking because I know so little about making good UIs, but it’s a start.

EJS

Used EJS for templating, and it’s great. Fast, flexible, and it’s not too bad that I have to fill in most of the HTML manually. I will know more about the structure of the client facing  side of the page more than I would otherwise.

Lessons learned

There are things that went well and not so well.

  • Maybe the most important: Facebook is unreliable. Sometimes I got missing people in the list that appear in the next update again. This happened when using Graph API, will have to see if FQL is any better. From my experience: not, it just has different problems.
  • For a while I started to write in Coffeescript, but then I just spent too much time trying to figure out how to get the right things, so went back to Javascript, at least until I learn that and can move up again to that higher level.
  • From the app’s point of view, some simplification needed. Too many middleware and helpers, this I quite often not sure about what is the state of my data, what variables were available and what are not.
  • Forums are not always helpful, quite often people have really dumb answers to questions, so have to figure things out myself.
  • Too many successful libraries mean that the library-soup will inevitably cause problems, and people start writing new libraries that connect and unify the old ones: mongoose & backbone, mongodb & everyauth, so much magic (ie. behind the scenes stuff) happens.
  • I still like to make logos and facicons, though I’m sure they are terrible, it makes good playtime in Inkscape and Gimp. Might have to change the colours, it does remind me of a strange toilet logo…
Friendcare logo

Aftermath

In the end, I’m solving a very narrow problem at the moment, a problem that so rarely arises, that I don’t even know if it will happen again. But if it does happen, without this I wouldn’t know about it, so still worth it.

Might extend it later to count Twitter followers, Google+ circles, Github watch/fork, whatnot… But that’s later when I see how does this work, I care much less about those than my “friends” because of the symmetricity of it .

The good thing is that currently it works well enough that I kinda got this off my chest and can back to making a FUSE module for Ge.tt, a group based location sharing mobile app, an app to connect my Goodreads to-read list and bookstore databases,  adding Open Graph and Rich Objects to Octopress, getting out an update for WatchDoc, or even check out the games I just got from HumbleBundle. But will come back to this, will have to make it better (it’s a compulsion).

Finally, if you the site, let me know your suggestions, I’m eager to learn. The source is up on Github.