try all your chance

Shenzhen I/O Tetris

Sat, 22 Oct 2016 00:00:00 +0000

Background

One of the missions in Shenzhen I/O is to create a game in the sandbox.

Intimidated by the idea of drawing something custom for the LCD screen, I thought about making Tetris with the default block matrix.

As that is obviously impossible, I went to skip the mission.

Just before the bar filled up, I stopped. The text on this button believed in me. Could I really betray it? Could I sit there stone faced, lying through my mouse finger to it? The JRPG heroes and moral-of-the-story-having cartoons and sitcoms of my childhood raised me better than that.

There was only one honorable thing to do: a hard work montage.

There should be a video here!

Phew! Now, let me ruin the pace we’ve set by explaining everything in excruciatingly boring detail.

Major challenge: storing more than one anything

Processors have only one working register: acc. The larger ones add dat, which can only be set with mov. That means to perform operations on the value of dat, you must clobber acc, so even the big processors can only effectively store a single value.

mov dat acc
add 1
mov acc dat

dgt and dst

dgt and dst make it possible to isolate individual digits. Registers can hold three digits.

A falling tetris piece has three values that change while it’s falling: the column (X), the row (Y), and the rotation.

There are only 4 possible rotations for a piece, so that’s easy to fit in a digit. The default LCD matrix is 10x12, so X already fits in a digit, and Y can fit in a digit if we give up the bottom two rows.

Thus we can store our whole state in one register: the left digit is the rotation (0-3), the middle digit is the row (0-9), and the right digit is the column (0-9).

To move a piece right or left we just add 1 or sub 1, to move it down we add 10, and to rotate it we add 100.

To convert this number to a cell on the matrix, we remove the rotation (dst 2 0) and add 1.

Digits as bits: it’s all 1s and 2s

We can represent all but the line piece with 6 bits. Using 1 for “on” and 0 for “off” is natural, but using 2 and 1 makes it easier to decode.

Notice how most pieces share a row with another piece. This lets us squish them together when we store them in a ROM chip. This did not end up being useful, but it is neat.

We can step through these bits different ways to render different rotations:

Bits as bits

To determine when you’ve run into a piece that’s already in place, we need one bit for every space on the board. Since we decided the board is 10x10, that means we need 100 bits of storage.

The RAM module has 14 cells, which multiplied by 3 digits give us 42 bits of storage. Even if we used two we don’t have enough space to store the whole state of the board.

If we had some way to use binary numbers, we could store three bits per digit using values 0-7, which would give us 9 bits per cell for 126 total bits of storage.

If we then give up a column on the board, each cell can store a whole row. This lets us use one digit to pick a memory cell and one digit to pick a bit, and makes clearing completed lines very easy.

There’s no div instruction, but we can divide a single digit by 3 with mul 34 and then dgt 2.

To convert the digit 0-7 into its component binary bits, we can multiply by 125. A set bit is indicated by a digit greater than 3 in the result. Since it is not any more expensive for callers to tgt reg 3 than teq reg 0, we don’t need to interpret it here.

Putting it all together

Download the working game here. Left and right move, up rotates. The game ends when it crashes. There’s still plenty of room to optimize and add more. The board is laid out roughly like this:

Here is a video of it in action:

Mopidy with Extensions on NixOS

Sun, 20 Mar 2016 00:00:00 +0000

Mopidy is a sweet mpd-compatible music player that can play local files and stream from services like Spotify and YouTube.

NixOS has a package for Mopidy and several extensions. You can install them with nix-env -i, but Mopidy will not detect the extensions, and it can be difficult to figure out why.

This is Nix working the way it is supposed to work: the extensions are Python libraries, and the build environment for the mopidy package can’t be altered by installing other packages.

The solution is to make a new environment that depends on the mopidy package and any extension packages you want.

It’s easy with nix-shell:

nix-shell -p mopidy mopidy-mopify mopidy-spotify --run mopidy

Turned into an executable script:

#! /usr/bin/env nix-shell
#! nix-shell -p mopidy mopidy-mopify mopidy-spotify --run mopidy

There is also a mopidy service, used with something like this in /etc/nixos/configuration.nix:

services.mopidy = {
  enable = true;
  extensionPackages = [ pkgs.mopidy-spotify pkgs.mopidy-mopify ];
  configuration = ''
    [spotify]
    username = dade.murphy
    password = hack.the.planet
  '';
};

XMonad, Spacemacs Style

Sat, 19 Mar 2016 00:00:00 +0000

Background

I’ve been using XMonad for almost 10 years but I barely know my own key bindings. Every time I open xmonad.hs I find bindings I don’t remember setting, doing things I’d forgotten were even possible. I’m hesitant to add new bindings because I know I’ll forget them.

As a recent convert to Spacemacs, one of the things I love most is that the key bindings are mnemonic key sequences, with help that pops up when you hesitate along the way.

I want the same thing in XMonad.

Making It So

Key Sequences

The first problem was figuring out how to bind key sequences instead of key presses. I searched for things like “xmonad key sequence,” “xmonad multiple key bind,” “xmonad prefix key,” and so on, until finally a blog post by Alexander Kojevnikov pointed me in the right direction.

XMonad.Actions.Submap lets you bind a key to the submap action, which accepts another map of key bindings. When called, submap intercepts the next keypress and dispatches accordingly.

Even Better: Emacs-style Bindings

From Submap I stumbled onto XMonad.Util.EZConfig, which makes the key binding syntax much nicer. Instead of specifying a tuple with the modifier bitmask and appropriate xK_ variable, you just use a string with an Emacs-like notation. It even supports specifying a sequence of keys, so you don’t need to use Submap directly.

-- Using only Submap
((modm, xK_x), submap . M.fromList $ [((0, xK_w), spawn "xmessage 'woohoo!'")])

-- With EZConfig
("M-x w", spawn "xmessage 'woohoo!'")

Visual Feedback

Submap and EZConfig don’t provide any kind of feedback that you’ve entered a submap, or expose any way to add your own. We can use >> to call spawn before submap, but submap blocks waiting for input; dmenu won’t run because it can’t grab the keyboard, and xmessage won’t appear because XMonad won’t notice the new window until after submap exits.

Luckily, dzen2 works and is perfect for what I have in mind. We just have to go back to calling submap manually:

mySubmap = submap . M.fromList $
  [((0, xK_w), spawn "xmessage 'woohoo!'")
  ]

main = xmonad $ defaultConfig `additionalKeys`
                [ ("M-m", spawn "echo Entered Submap | dzen2 -p 2"
                       >> mySubmap) ]

With no parameters, dzen2 closes when its input is closed. We can use -p 2 to keep it open for 2 seconds, but what we really want is for it to wait a second before appearing, and to disappear when we exit the submap. With spawnPipe from XMonad.Util.Run we can get a handle that we can write to and close with System.IO:

mySubmap = submap . M.fromList $
  [((0, xK_w), spawn "xmessage 'woohoo!'")
  ]

dzen message action = do
  handle <- spawnPipe "sleep 1 && dzen2"
  io $ hPutStrLn handle message
  action
  io $ hClose handle
  
main = xmonad $ defaultConfig `additionalKeys`
                [ ("M-m", dzen "In Submap" mySubmap]) ]

(io is necessary to lift the IO monad into the X monad, because Haskell)

Parsing the Keymap

Now we can pop up a message, how do we make it tell us what the keys do?

XMonad.Util.NamedActions is an option, but it looks really cumbersome. I don’t want to have to think of documentation and keep it up to date. It might be possible to do something with Template Haskell to auto-generate names for actions, but that seems like too much work.

Since I keep my bindings neatly formatted anyway, the easiest thing to do is just yank the keys and actions directly out of xmonad.hs.

I know my submaps are always going to start with a line like submapName = and end with a ] on a line by itself. awk makes it easy to grab a range of lines:

~$ cat ~/.xmonad/xmonad.hs | awk '/^mySubmap/,/]$/'
mySubmap = submap . M.fromList $
  [((0, xK_w), spawn "xmessage 'woohoo!'")
  ]

With a little regular expression magic, sed can extract the import parts:

~$ !! | sed -r '1d;$d;s/.*xK_(.)\), (.*)\)$/\1 -> \2/'
w -> spawn "xmessage 'woohoo!'"

(-r swaps which kind of paren we need to quote, and 1d;$d trims the first and last line)

With a bit of work you could do this with either sed or awk alone, but I think this way is simpler.

Later, once we have a bunch of them, column can wrap them into columns:

~$ for i in {1..6}; do !!; done | column
w -> spawn "xmessage 'woohoo!'"	w -> spawn "xmessage 'woohoo!'"
w -> spawn "xmessage 'woohoo!'"	w -> spawn "xmessage 'woohoo!'"
w -> spawn "xmessage 'woohoo!'"	w -> spawn "xmessage 'woohoo!'"

Now instead of calling dzen2 directly, we can call a shell script that contains this command pipeline:

#!/bin/sh
KEYMAP=$1
( echo "$KEYMAP"                                   \
; cat ~/.xmonad/xmonad.hs                          \
  | awk '/^'"$KEYMAP"'/,/]$/'                      \
  | sed -r '1d;$d;s/.*xK_(.)\), (.*)\)$/\1 -> \2/' \
  | columns | expand
; cat ) | dzen2 -l 5 -e onstart=uncollapse

A few things to note:

columns outputs tab characters, which don’t work in dzen2. expand converts them to spaces.
We need to run the pipeline in a subshell that ends with ; cat to connect dzen2’s input channel to the output handle we have in XMonad.
To make dzen2 show multiple lines, we need the -l parameter. This makes the first line the title, and the rest show up in an additional window that is hidden by default unless we add -e onstart=uncollapse.

Positioning

The last piece of the puzzle is getting the help to popup on the bottom of whatever monitor has focus, and dzen2 doesn’t seem to have any easy way to do this. It does make it easy to manually set the location and dimensions, we just need to figure out what those are.

We can get the location and dimensions of the focused screen in XMonad by stealing some code from floatLocation in XMonad.Operations:

windowScreenSize :: Window -> X (Rectangle)
windowScreenSize w = withDisplay $ \d -> do
    ws <- gets windowset
    wa <- io $ getWindowAttributes d w
    bw <- fi <$> asks (borderWidth . config)
    sc <- fromMaybe (W.current ws) <$> pointScreen (fi $ wa_x wa) (fi $ wa_y wa)

    return $ screenRect . W.screenDetail $ sc
  where fi x = fromIntegral x

focusedScreenSize :: X (Rectangle)
focusedScreenSize = withWindowSet $ windowScreenSize . fromJust . W.peek

We can then pass those to our script with something like:

keyMapDoc :: String -> X Handle
keyMapDoc name = do
  ss <- focusedScreenSize
  handle <- spawnPipe $ unwords ["~/.xmonad/showHintForKeymap.sh", name, show (rect_x ss), show (rect_y ss), show (rect_width ss), show (rect_height ss)]
  return handle

The math is straightforward so I won’t cover it here. We end up with something like:

LINE_HEIGHT=14
INFO=$(cat ...)
N_LINES=$(wc -l <<< "$INFO")
OFFSET=$((LINE_HEIGHT * N_LINES))
Y=$((BOTTOM - OFFSET))
(echo "$KEYMAP"; echo "$INFO"; cat) | dzen2 -h $LINE_HEIGHT -y $Y ...

The Result

It’s not quite nice enough to start a project called spacenads yet, but it’s exactly what I wanted. I have made some more tweaks and gone a little further than is described here, but you can see what I’m actually using on my github.

Groovy Gotchas

Thu, 13 Oct 2011 03:16:32 +0000

Map[‘key’]

When calling putAt() on a Map with a String as a key, the Object version of putAt() is selected over the Map version. In other words, it will only call Map.put() if a property with the same name does not exist. However, calling getAt() on a Map will only call Map.get(), and will never return an object property.

class A extends HashMap { String y; } // any Map

a = new A();

a['x'] = 1;
a['x'];           // => 1
a.get('x');       // => 1
a.putAt('x', 2);
a.getAt('x');     // => 2

a['y'] = 1;
a['y'];           // => null
a.get('y');       // => null
a.putAt('y', 2);
a.getAt('y');     // => null
org.codehaus.groovy.runtime.DefaultGroovyMethods.putAt(a, 'y', 3)
a.getAt('y');     // => 3 !?

class B { String y = "hi"; }
(new B()).getAt('y');  // => "hi"
(new B())['y'];        // => "hi"

class C implements Map { String y; /* implement Map methods */ } 
(new C()).getAt('y');  // => null
(new C())['y'];        // => null

Ranges

The exclusive range operator (..<) generates a Range where the ending value is one step closer to the beginning value. The less-than symbol can be somewhat unintuitive for descending ranges.

enum E { ONE, TWO, THREE, FOUR }
((E.ONE)..(E.THREE)).toList()  // [ONE, TWO, THREE]
((E.ONE)..<(E.THREE)).toList() // [ONE, TWO]
((E.THREE)..(E.ONE)).toList()  // [THREE, TWO, ONE]
((E.THREE)..<(E.ONE)).toList() // [THREE, TWO]

Indexing a List with a Range only considers from and to values after the above adjustment; the result of Range.toList() is irrelevant, and whether or not it was an exclusive range is not known. Three steps are performed to get the result:

Negative values are normalized to positive values by adding them to List.size()
The result is generated from List.subList( min(from, to), max(from, to) )
If from > to, the result is reversed

In Ruby, a[0...-1] means “From the 0th element up to and excluding the last element,” whereas the ostensibly equivalent construct in Groovy, a[0..<-1], means “From the 0th element to the 0th element.”

a = [0,1,2,3,4]

a[0..-1]             // => [0, 1, 2, 3, 4]
a[0..-2]             // => [0, 1, 2, 3]
a[0..<-1]            // => [0]
a[0..<-2]            // => [0, 1, 2, 3, 4]

a[0..-2]             // => [0, 1, 2, 3]
a[-1..-2]            // => [5, 4]
a[-1..<-2]           // => [5]

Compare to Ruby:

a[0..-1]              # => [0, 1, 2, 3, 4]
a[0..-2]              # => [0, 1, 2, 3]
a[0...-1]             # => [0, 1, 2, 3]
a[0...-2]             # => [0, 1, 2]

a[0..-2]              # => [0, 1, 2, 3]
a[-1..-2]             # => []
a[-1...-2]            # => []

TIP: Fixing author in git history

Sat, 23 Oct 2010 22:34:36 +0000

I always forget to set up my user info on git on new machines before I check stuff in. It’s pretty easy to fix if there’s nobody else in your repo:

git filter-branch --env-filter "\
  export GIT_AUTHOR_NAME=Dade\ Murphy \
         GIT_AUTHOR_EMAIL=zer0cool@example.com \
         GIT_COMMITTER_NAME=Dade\ Murphy \
         GIT_COMMITTER_EMAIL=zer0cool@example.com "

Source: serverfault

Starcraft vs Monty Hall

Sun, 02 May 2010 21:46:11 +0000

If you’re not familiar with the Monty Hall problem, it goes something like this:

There are three doors, and one of them has a prize. You choose one of the doors, and Monty opens one of the others that is not a winner. Now you have the option to stick with your original choice, or switch to the remaining door. It might seem counter-intuitive, but switching doubles your odds of winning.

Some people have attempted to apply the same logic to scouting as Zerg using an overlord and a drone in Starcraft. Stated similarly, the problem goes like this:

There are three starting locations, and one of them has your enemy base. You send a drone to one location, and your overlord gets to one of the other locations and discovers no enemy base. Now you have the option to stick with your original choice, or send your drone to the remaining location.

Sounds the same, but in this case, switching has no effect on your odds of winning. It is the same as the “Monty Fall” or “Ignorant Monty” variant of the Monty Hall problem, where Monty opens a door completely at random rather than one which is a non-winner.

The difference is because in the classic Monty Hall problem, you are initially choosing one door, which gives you 1/3 odds of your first choice being right. If you could choose to switch to both other doors, you’d obviously have a 2/3 chance of winning. In fact, this is exactly what you are doing when you switch, even after one of the doors has been revealed.

In the Starcraft problem, you are choosing two locations to begin with, which gives you a 2/3 chance of being right. If you choose to switch your drone to the remaining location at any point, the overlord still has a 1/3 chance and the drone still has a 1/3 chance. In the Monty Hall problem, you switch from only an unknown door (1/3), to the empty door and an unknown door (2/3). In the Starcraft problem, you switch from both an empty location and an unknown location (2/3), to the same empty location and a different unknown location (2/3).

To demonstrate this visually, I’ve made a simulator in Javascript.

Update (5/3): I added a “Monty Hall mode” to the simulator, the implementation of which may help make this even clearer. Normally, I choose two random locations from the possible enemy starting points, and send the overlord to the first and the drone to the second. This leads to 3 possibilities with an equal chance of occuring: the overlord finds the base, the drone finds the base, or neither finds the base. Only in the final case, which occurs 1/3 of the time, would it be correct to switch. In “Monty Hall mode,” the overlord is not allowed to find the base - so the 1/3 of the time where the overlord would normally have found the base, is now added to the 1/3 of the time when neither finds the base, making it correct to switch 2/3s of the time.

I also added a “Stupid Overlord mode” which demonstrates that the order is important. If the overlord chooses an empty base first, rather than the drone choosing first, the chances of the drone being correct are (obviously) 1/2.

Mixing Up Code and Data

Sat, 01 May 2010 08:12:56 +0000

From buffer overflows to cross-site scripting, decades of software security flaws can be traced back to a simple design problem: executable code (or otherwise specially meaningful data), and non-executable, black-box data are intermingled in the same channel. To execute arbitrary code, traditional buffer overflow exploits rely on non-executable data trampling execution state and eventually causing data to be executed as code. Cross-site scripting exploits, and all traditional injection exploits, work when intermediary systems fail to identify the difference between code and data in exactly the same way as some other system.

To prevent these exploits, developers are generally advised to canonicalize input and encode data in output. While this is correct, I think it is important to also understand that it shouldn’t be that way. Developers should have to go an extra mile to cause data to be interpreted as code, not the other way around. Consider this common SQL injection scenario:

$result = mysql_query( "select id, name, pass from users where name='$name'", $db );

While it’s clear that $name is simply data to the developer, it’s being crammed into a string along with code and handed off to another system for re-interpretation. A common solution to prevent injection is something like this:

$query = sprintf( "select id, name, pass from users where name='%s' ", mysql_real_escape_string($name) );
$result = mysql_query( $query, $db );

It’s true that this solution fixes the vulnerability in this instance, but it’s a heavy burden: always remember to add a whole bunch of code every time you perform this incredibly common task, or you will introduce one of the most serious vulnerabilities possible into your application. In my opinion, even though the code may be “secure,” it is still wrong. You are still mixing up code and data in the same place on the same channel. Here is a better solution:

$stmt = $mysqli->prepare("select id, name, pass from users where name=?");
$stmt->bind_param("s", $name);
$stmt->execute();

This looks similar, but it’s important to understand how it works. The first line sends all of the code to be parsed and prepared by the server. Data to be filled in later is represented by question marks, which can only be used where data is expected. After this point, nothing is re-interpreted. When we send the data on the second line, the server is only reading data, and no special encoding is required to prevent it from being treated as code.

Even though it suffers the same criticism of being extra code every time you want to execute a SQL query, a developer used to coding this way is much less likely to make a mistake than a developer used to the previous example. The statements could also be prepared in an initializer somewhere completely different instead of right next to where data is being bound, to make it even harder to make a mistake.

Imagine if HTML/HTTP had something similar:

<html>
<div class="title">&data[0,16];</div>
<div class="body">&data[17,256];</div>
</html>
Data until EOF. <!-- &amp; No possibility of being interpreted as html. <script>alert(1);</script>

Note that the data part is a complete black box and terminated out-of-band (by closing the connection). There aren’t lengths or other special markers embedded in the data. I’m not suggesting this is a realistic possibility for HTML or any other established standard. I just want to make the point that a protocol which makes it impossible to confuse code and data is invulnerable to injection attacks by default.

A related idea is identifying the source of data. When an application includes user input in its own output, or in SQL statements, or anywhere else, it takes responsibility for it; to other applications, it’s data (or code) coming from the first application, not data coming from a user or third-party application somewhere else.

In cases where user input needs to be specially interpreted, especially by some other application - when you want to let people include a little HTML in a post, for example - then you have a more difficult problem which can’t be solved by shoving it all into a separate black box. But imagine how nice it would be if you could specify that some bit of HTML was from a user, or some other website (perhaps with digital signatures), or whatever, and the browser could handle sandboxing and cross-domain policies and everything else it already has the ability to do.

I think it is important to address this idea up front not just when designing a protocol or file format, but when designing APIs and internal interfaces to existing systems. Even if you have to output HTML or actual SQL statements, an interface to generate the output which cleanly separates code and data can relieve the burden from developers, isolate potential weaknesses in one place, and go a long way to improve the security of a system.

TIP: Using GDB as an Interactive C Shell

Fri, 26 Mar 2010 20:13:54 +0000

Many programming languages come with some way to run an interactive shell, or REPL (read-eval-print loop). This makes it extremely easy to test little bits of code and understand exactly what they do, and is invaluable when learning a new language or library. For example:

What’s the result of (unsigned int)atoi("4294967295") in C?

Even if you know the answer, how quickly can you prove it? How concisely can you communicate the proof via IM or email? What if it’s a poorly documented third-party library function, and not a standard one?

For quick tasks, you can just use gdb which is probably already present on any system that has gcc. Just fire up gdb on any binary, set a breakpoint on main, and run. When it stops you will be able to call functions and examine their results, and many other common REPL tasks. The binary doesn’t matter much, but you should prefer ones with debugging symbols, and if you want to call functions in a particular library, you should use a binary that is linked to that library.

Example session:

~% gdb ./test
(gdb) break main
Breakpoint 1 at 0x8048452
(gdb) run
Starting program: /home/pcl/sandbox/test
Breakpoint 1, 0x08048452 in main ()
(gdb) set $a = malloc(1234)
(gdb) call sprintf($a, "Hello %d", 12345*12345*12345)
$1 = 15
(gdb) print (char*)$a
$2 = 0x96c6008 "Hello 170287977"
(gdb) print (unsigned int)atoi("-1")
$3 = 4294967295
(gdb) print (unsigned int)atoi("4294967295")
$4 = 2147483647

gdb lets you use arbitrarily-named, untyped convenience variables, as you can see in the example. The only practical difference between print $var = expr, call $var = expr, and set $var = expr seems to be that set does not additionally assign the result to a history variable. Obviously you also have the full debugging facilities of gdb available as well.

It is also possible to do this on stripped binaries with no ‘main’ function, but there are many disadvantages:

~% gdb `which echo`
(gdb) inf files
	Entry point: 0x8048be0	0x08048154 - 0x08048167 is .interp
(gdb) break *0x8048be0
Breakpoint 1 at 0x8048be0

For a fully featured REPL for C, check out c-repl.

Analysis of Gemini Cybernetics CDS

Thu, 25 Mar 2010 03:13:55 +0000

There have been some rumors going around about a new third-party system for Second Life. The system attempts to detect avatars using third-party clients capable of duplicating objects without the creator’s permission, and the rumors are that it uses some kind of QuickTime exploit or other nefarious means to actually examine the contents of your hard drive or otherwise invade your privacy without permission. I decided to take a quick look to see what it’s all about.

The system in question is called GEMINI CDS Ban Relay and is advertised as a simple object which detects avatars entering your sim, and uses “a team of bots with special abilities” to determine if the avatar is “harmful.” If they are, it adds them to an external database, and can optionally ban or teleport them home. Entries in the database are permanent, so if an avatar has been considered harmful once, they are always considered harmful in the future. It claims to use several frequently updated methods to detect “illegitimate” clients.

The most obvious detection method, and the only one I discovered, is a script that triggers as soon as you enter a protected sim and tells your client to load up a special media URL. Using a tool like Wireshark or ngrep, it is trivial to watch the HTTP request.

Broken down, the requested URL in my case was:

http://media.syscast.net/youtube.php
  ? licensekey = KBVaQkxGH1lDVRdBWA1GVEdaTFpQF1ReWUcREU9YEFxBRgxE
  & title      = BEYeQR8TAxxOLE8eBk0T
  & licensedon = B1IAQhUG
  & tvowner    = eBVbGUdLQFxeVA%3D%3D
  & videoid    = eEJVE0xDHwxZAUQSWBJFARMJV1RTE19BWUETGBMNQg0%3D

At a glance, the values are obviously all Base64 encoded – the trailing %3Ds on the last two fields are a dead giveaway. Decoding them doesn’t produce anything human-readable, though; one online service gives me “(ZBLFYCUAXFTGZLZPT^YGOX\AFD” for the first field.

It’s easy to decode them in Ruby, where we can play with them a little more:

irb(main):002:0> CIPHERTEXT = Base64.decode64('KBVaQkxGH1lDVRdBWA1GVEdaTFpQF1ReWUcREU9YEFxBRgxE')
=> "(\025ZBLF\037YCU\027AX\rFTGZLZP\027T^YG\021\021OX\020\\AF\fD"

irb(main):003:0> CIPHERTEXT.length
=> 36

36 bytes is the same length as a UUID in canonical form, so it’s a pretty reasonable guess that this is my avatar’s UUID encrypted somehow. The only real encryption facility LSL makes available is XORing Base64-encoded strings together. XOR has an interesting property: a ⊕ b = c ⇒ a ⊕ c = b; that is, if XORing some plaintext and some key produces some ciphertext, then XORing that ciphertext and the plaintext produces the key. Let’s give it a shot:

irb(main):004:0> PLAINTEXT = 'a27b84f0-2757-4176-9579-43a181d4a5a0'
=> "a27b84f0-2757-4176-9579-43a181d4a5a0"

irb(main):007:0> CIPHERTEXT.bytes.each_with_index {|v,i| key << (v ^ PLAINTEXT[i])}; key
=> "I'm trying to replace msmtp with smt"

That was easy enough. Using the key to decode the rest of the fields, we can see what is really being sent:

http://media.syscast.net/youtube.php
  ? licensekey = a27b84f0-2757-4176-9579-43a181d4a5a0
  & title      = Masakazu Kojima
  & licensedon = Numbat
  & tvowner    = 1269399503
  & videoid    = 1e8381fe7fdf727dce67632245c8dd6e

The second field (title) turns out to be my avatar name, followed by the sim name (licensedon), a UNIX time value (tvowner), and what looks like an MD5 hash (videoid). The time value is apparently used to prevent replay attacks: it is possible to immediately replay the request exactly and get a success response, but after about 30 seconds it causes an internal server error instead.

Visiting the parcel again to get another URL shows that only the time and MD5 hash change. Tampering with the values causes an immediate error redirect, which suggests that the MD5 hash is a signature to prevent forged messages. So, even though we could encrypt arbitrary values and send them, we’d need to know how the signature is generated for them to work.

The response from the server is innocent enough:

<!--
-->
<html><head></head><body bgcolor="#7f7f7f" leftmargin="0" topmargin="0"><img src="video-background.gif" width="2000px" height="2000px" border="0px" /></body></html>

The video-background.gif file is just a transparent 1x1 GIF image:

00000000  47 49 46 38 39 61 01 00  01 00 80 00 00 7f 7f 7f  |GIF89a..........|
00000010  00 00 00 21 f9 04 00 00  00 00 00 2c 00 00 00 00  |...!.......,....|
00000020  01 00 01 00 00 02 02 44  01 00 3b                 |.......D..;|

These are the only requests that are performed. Nothing nefarious appears to be taking place. There is no evidence of any kind of exploit, or the transmission of any kind of private information. So how does the service detect “illegitimate” clients?

The magic turns out to be in the “User-Agent” request header, which identifies the client. In my case: Mozilla/5.0 (Windows; U; Windows NT 6.0; chrome://navigator/locale/navigator.properties; rv:1.8.1.21) Gecko/20090305 SecondLife/Emerald Viewer (default skin)

By using curl to replay an old request, and simply replacing “Emerald Viewer” with the name of a random client from the Onyx project (NeilLife), I was able to get the system to ban an alternate account I created. Note that this worked even though the time value was old, and the HTTP response status was a 500 error, so it would appear that the system to prevent replay attacks is broken. Looks like they’re up to at least 1 false positive, even if it’s a technicality. Also note that the actual response body did not change, so there doesn’t seem to be any kind of exploit that is only sent to users of “bad” clients.

Using the same IP address and computer, I was able to go back to the same parcel on my main account with no trouble.

Conclusions

Despite all the subterfuge, the Gemini CDS system seems to simply rely on “illegitimate” clients to identify themselves in an HTTP request. The message encryption is trivial to break, and it would seem that is only a matter of time before someone cares enough to figure out how to forge the message signature. It is trivial to avoid detection by this method, though the system may employ additional detection methods: the common way for third-party clients to identify each other is by the unique texture UUIDs they use for skin layer protection, which is passive and undetectable.

There is no evidence that the system uses any kind of exploit or other nefarious tactic. The system does not appear to record any data other than the avatar and client self-identification information.

TIP: Make bash tab completion ignore .svn directories

Tue, 16 Mar 2010 15:50:45 +0000

Having to tab through the fifty million otherwise empty “net/mycompany/project/unit/subunit” directories that the Java ecosystem necessitates has consistently driven me crazy because completion stops at each step to let me choose the .svn directory, and I have to look and type the first letter of the directory I actually want to make it continue.

It’s actually really easy to fix this:

export FIGNORE=.svn

$FIGNORE is just a colon-separated list of suffixes to ignore when doing tab completion.