There are 26 maps in the collection, made by the US Army by flying over different parts of the island, and mostly I guess stitching together aerial photographs. The maps themselves were not that easy check in an image viewer, since there’s no context, zoom is clumsy, and have no idea where about half the places should be located. Instead, I thought it would be great to have them as an overlay on top of current maps and satellite imagery on Google Maps.

The result is Taiwan City Maps overlays, which does exactly that. Feel free to click the link and explore right now! In the rest of this post, I try to first show how that page was made, and also some history lessons I gained by making it.

How it was made

The Google Maps Ground Overlay API does exactly what I needed, so can’t claim much (almost any) innovation in the page, though I stumbled on it after a few other “Overlays”, as Google Maps has a couple, all working slightly differently.

Ground Overlay basically takes the following parameters:

• the map’s center,
• zoom level,
• an image,
• the image’s N/E/W/S bounds in geographical coordinates.
• whether to show satellite or maps view underneath

Even cooler, the opacity of the image can be adjusted, so the overlay and the base layer can be viewed together much easier!

In that list above, the trickiest part is of course figuring out the boundaries of the image – basically the way to overlay them – for multiple reasons. First, all the maps have borders with information, titles, legend, and so on. Here below is the Taipei city plan from 1944. I can’t really tell where the corner of the image should be geographically, if there’s nothing to compare to.

Then there’s also the problem of things have changed in the last ~70 years, so a lot of things are different on the maps, if I want to match them up. Lastly, the maps are not perfectly accurate either, they are stitched from multiple photographs of unknown quality.

Been trying to match the maps by hand for a while, and codings some tools to make that process easier – but it didn’t really get easier. I needed to get to get around all these issues, and have a reasonable, objectively good fit. Objectivity is a key, and math helps with that. What I ended up with doing a linear fit of geographical and image coordinates.

• look for landmarks that are reasonably the same back then and now
• note their geographical coordinates, and their pixel coordinates on the image
• note the image size
• finally: run a linear fit that gives back the geographical coordinates of the image edges that would best match the landmark data in both sources

In really ugly Python it would be something like this:

import numpy as np

# # For example the contents of "taoyuanfit.csv":
# # Landmark data: image vertical/horizontal (px), latitude/longitude
# 1677,2861,24.991962,121.322634
# 1597,524,24.992282,121.304856
# 2385,1348,24.986873,121.311122
# 764,1342,24.998116,121.311411
datafile = "taoyuanfit.csv"
points = np.loadtxt(datafile, delimiter=",", comments="#")
imgwidth, imgheight = 3975, 3209

# Points on the image, scaled to range 0 to 1
px, py = points[:, 1]/imgwidth, (imgheight-points[:, 0])/imgheight
# Points on the map
gx, gy = points[:, 3], points[:, 2]
# Do the fitting
lx, ly = np.polyfit(px, gx, 1), np.polyfit(py, gy, 1)

# Fitting parameters give map boundaries, results in this case
west = lx[1]          # 121.300983
east = lx[1] + lx[0]  # 121.331141
north = ly[1]+ly[0]   #  25.003429
south = ly[1]         #  24.981204

This is now pretty objective, and then quality of the overlay depends on the quality of the landmarks I find, and the maps that they are on.

The best landmarks I could turn up were notable roundabouts like below…

… and recognizable streets and intersections, like below…

… but not too small and not too large streets. Large streets have fewer recognizable locations (unless some notable mid-sized street is intersecting with it), and small streets are usually not placed too carefully on the map, so can’t really trust their positions.

What I couldn’t trust, though: rivers (they changed a lot), bridges (usually can’t tell if they are the same bridge in both era, or a new one built close to the no longer there old one, as it is a common practice), coastlines, lakes, large structures.

The minimal number of landmarks needed for a fit (ideally) is just two, and surprisingly, there were a couple maps where I could get along with about three. In most cases, though, I needed at least 4-6 landmarks, and there were some I needed to discard for overall better quality of fit. Some maps show pretty good accuracy, while others (such as e.g. Kaohsiung) are quite poor – compared to today’s satellite pictures, of course. It’s all enjoyable, though.

Then added a few more functions to the page, such as changing the opacity, being able to link to the exact location and zoom level I’m looking at, and some page-load animation (the maps are in the range of 4-10MB, not that quick to load).

The source code for the site is on Github, and pull requests are accepted, naturally.

History lessons learned

I was very interested to note the things that have changed, and things that haven’t in the decades passed.

Things that mostly stayed the same

Schools are usually at the same location, even stayed the same type (e.g. girls’ high school staying that in Changhua).

Airports are usually at the same location (duh!), though usually greatly expanded, while the old runways can be matched to the new ones (like in Hsinchu).

Train stations, of course, although tracks might change, or entire stations disappear (like in Beigang or in Hualien).

Military installations, which surprised me, can stay the same, looks like the KMT troops just took over the Japanese posts and continued to use them. Some turned into schools, or could disappear too.

Many roundabouts, outlines of city blocks (like the CKS Memorial Hall now, or Yilan’s city outline), mountains, also stayed mostly the same.

Things that changed a lot

City areas changed enormously (duh!), just look at the capital, Taipei, the west side stayed mostly the same but built in about twice/three times as big area on the east. Most other cities grown just like that, as it’s very clear for example for Taitung. It’s no wonder, 5.8 million people on the island in 1940, while above 23 million by now (according to Wikipedia). This also brings the changes of a lot of streets and roads.

Water is definitely very changeable. Rivers got diverted a lot, most notably on the east side of Taipei (near Neihu and Songshan districts) the Keelung river was straightened up, but the roads can still mirror the old path. Other rivers (or channels?) got covered up, and now only live on as roads – though I guess they might still flow under our feet (just like Fleet Street)? Some more dramatic change is an entire missing island in Danshui, I wonder what happened to it? Lakes change too, mostly I guess due to development around them (like in Zuoying). The sea shore is different for the many cities, as probably they were developed and also the shallows were naturally filled by the rivers (like in Hsinchu and in Tainan). Harbors don’t stay the same either (like in Penghu).

Some landmarks such as shrines have changed too. The most notable I could find so far is in Taipei / Yuanshan, where the current Grand Hotel / 圓山大飯店 replaced a shrine in 1952 (yellow rooftop on the left below), while the Taipei Martyrs’ Shrine seems to be built on the site of another previous shrine in 1969 (yellow rooftop and cross-shaped grounds on the right below). On the map there’s another temple-complex looking structure in between them, that’s now the Taipei American Club and the International Community Radio Taipei (ICRT).

Race courses disappeared too – together with the cavalry, I guess (like in Tainan).

I’m sure there are a bunch of other things too, it will take some time to discover and understand things…

Other notes

The legends and information on the maps was pretty interesting. For example here’s the coverage map for Taipei / Taihoku, showing what aerial maps they stitched together, and when were the flights that took the photos:

The Internet also taught me a few things, such as this very insightful comment on Reddit that traced back the Chinese / Japanese city names of these maps.

I was also amazed by how detailed information the Americans had about the functions of the buildings – down to the type of factories, the type schools the buildings were. I would not be surprised if the maps were supplemented by some real intelligence from the ground. Especially likely as the big cities had so much more such details than the smaller locations (where there might not be more than a few likely military installations noted). I wonder if there’s any good source to learn about this – about American intelligence behind enemy lines in the Pacific theatre of WWII.

To the Future

Now that everything kinda works, I think I’ll spend more time looking at the maps instead of working with them, try to learn more.

Hosting this project on Github is definitely pretty easy and convenient, though I start to see that their speed is not necessarily very good for big assets like the map images. Could add Cloudfront very easily to speed things up, just I don’t necessarily want to pay for that extra speed just yet. :P

To be able to show the loading animation I currently need to use jQuery to preload the image, then rely on the browser caching to not to download the image for the second time when the image is attached to the map object to create the overlay. It works pretty well… except in the Facebook Android app’s built in browser – that happily ignores caching and downloads the image twice (ouch). Would be awesome to figure out how can I get directly from the Google Map API when it finished downloading the image.

Might revisit a couple of the maps to see if I can improve the fit, though really just a couple of them, most are totally fine for their quality. Should also add the Japanese writing of the city names for more authenticity and discoverability.

And in the meantime, keep checking the map, and any comment or feedback is appreciated! A big thanks to the University of Texas Libraries and the Perry-Castañeda Library Map Collection for the public domain maps!

The post Taiwan WWII Map Overlays appeared first on ClickedyClick.

Internet connectivity

One of the first concern is how do I get online on the go? There seem to be a bunch of companies providing mobile hotspot rental. The one I wanted to use originally is in Taiwan, and tried to arrange it too late (needs ~5 days in advance). Instead I found another company online called Japan Wireless. They can send a wireless hotspot or pre-paid 3G card to a hotel, or to an airport to be picked up. Since the hotspot I wanted to get was out of stock for the first half of the trip, I went with a 3G card and wanted to use an old Android phone to act like the hotspot.

Picking up the card at Kansai Airport was very straightforward. My Android phone worked much less. Might be getting “too old”, sometimes it couldn’t start the wireless sharing at all, though when it worked it was good for a while. It is also worth enabling “network traffic monitoring” and appropriate warning levels. The pre-paid card had 1G traffic included. Definitely does not recommend running a Play Store update while connected, apps can easily take up 30-50Mb, and 50% of your traffic allowance is gone before you can say “Ice Cream Sandwich”…

In the end the hotel had pretty good wifi, on the road the environment was more interesting than things online, and haven’t run into any emergency that needed online connection. I guess this kind of rental is good insurance, but can conceivably get by without it.

Navigation

One thing is going somewhere, and another is know where’s one going. A lot of plans were made in Google Map My Places. Restaurants to try, shrines to visit, shops to examine… Not wanting to rely on constant internet connection, and because My Places in Google Maps for Android is very much not user friendly, had to try another service.

Been using Maps With Me for a while that provides offline map access. The Maps With Me Pro version has KML import and fortunately My Places has KML export.

This solved 90% of the issues. Had places with names and details, and path. The pin types don’t show up (in Maps With Me all the pins are the same), I guess I’ll have to check if they are not included in KML, or something else goes wrong along the way.

Maps With Me Pro is one of the handful Play Store purchases I made, and I think this one totally worth that NT$180 ($6?).

Being able to see the map is good, but it helps if the location is locked on quickly. GPS always takes a while on my phone, and to speed this up, I thought I just run GPS all the way… MyTracks worked for that pretty well, it recorded where we went, and helped to quickly see on Maps With Me as well where we are.

Of course it drains the battery, but being in Airplane Mode and only having the GPS on I could get about 10-11 hours out of the phone, which was okay for the day – provided that I haven’t browsed much on the portable hotspot. :)

Photography

Between the two of us on the trip we had 5 cameras (2 cell phones, a compact camera, a DSLR and a GoPro). Based on experience on other trips, it is pretty difficult to synchronize the clocks of all these cameras, so in the end I could have a real timeline.

The phone synchronized itself just by connecting to the Japanese cell network for a moment. GoPro was synchronized by the GoPro app. That leaves the other two cameras, that needed manual adjustments, so inevitably there were few seconds differences. It also doesn’t help that there’s 1h timezone difference between Japan and Taiwan, so had to set the time ahead and back on the plane, so that it will be correct in the local time zone.

It is still troublesome, and neither Facebook nor Google+ handled the time ordering completely correctly in the end when I was uploading the resulting pictures, selected from all devices. Google+ was a bit closer to be correct and could order in a second. Facebook sometimes was very much off and the ordering hung Chrome for a minute or so…

I guess manual processing would work, using a script (used Python for this before but not now) to turn timing information into filename information, then the ordering can be fixed easier and all the platforms handle that. In the same time, I’d love to see whether there’s a technical solution to synchronize time across cameras easier. I cannot be the only one who need that.

The phone camera had one more usage. HTC has pretty decent HDR (as point-and-shoot HDR pictures go), and could get a bit better scenes out of the very contrasting light occasionally. Though more often it was the “Overly Dramatic Photoshoot” effect. That’s fun too.

The cell phones were backed up over wifi during the trip, would want to figure out in the future how to back up the SD cards of the cameras as well, without a computer.

Communication

Only used a few times, but it does help to know a bit of Japanese. Tried to find any useful apps in the phrasebook category (that’s the lazy student’s solution). There were some good ones, but there was a surprising problem: I was looking for shopping related terms, and for every single app, shopping is a premium feature.

Unlike Maps With Me, I couldn’t find any travel phrasebook that I would want to upgrade to pro. So that’s all, got along with some English and lots of gestures. For the future I will be looking for some proper Japanese learning app instead of the quick solution.

Airport

China Airlines offered online check-in and also a Mobile Boarding Pass. It’s a PDF, and I have saved in Evernote, where I have offline access to it. Supposed to be very handy, though at the very first time it was used by the security at the Taoyuan Airport it kinda failed already: the security guard couldn’t scan my QR Code at all. No amount of resizing nor brightness adjustment worked. Holding up the entire queue did make me sad a bit, fortunately the other scanners seemed to have worked.

The immigration section of the airport was pretty techy as well. For the last few times I’ve been using the Taiwanese e-gate immigration: scan the residency permit (or passport for the Taiwanese), step in a little booth, run a facial recognition, if works let you out, if not then do a fingerprint check and continue. For me it’s the fastest immigration ever: facial recognition just flashes on the screen and lets me through right away. Probably not that many Western guys in the databases so it’s easy to verify me.

Next time

Some of these trick I will like to keep for next time I travel, though I do feel sometimes that technology that supposed to make my life easier and more smooth, when combined with other tech can become overwhelming.

Would love to hear about useful travel tech you are using as well, any suggestions to me?

The post My IT travel pack appeared first on ClickedyClick.