Groovy Gotchas

October 12, 2011 by · Leave a Comment 


When calling putAt() on a Map with a String as a key, the Object version of putAt() is selected over the Map version. In other words, it will only call Map.put() if a property with the same name does not exist. However, calling getAt() on a Map will only call Map.get(), and will never return an object property.

class A extends HashMap { String y; } // any Map

a = new A();

a['x'] = 1;
a['x'];           // => 1
a.get('x');       // => 1
a.putAt('x', 2);
a.getAt('x');     // => 2

a['y'] = 1;
a['y'];           // => null
a.get('y');       // => null
a.putAt('y', 2);
a.getAt('y');     // => null
org.codehaus.groovy.runtime.DefaultGroovyMethods.putAt(a, 'y', 3)
a.getAt('y');     // => 3 !?

class B { String y = "hi"; }
(new B()).getAt('y');  // => "hi"
(new B())['y'];        // => "hi"

class C implements Map { String y; /* implement Map methods */ } 
(new C()).getAt('y');  // => null
(new C())['y'];        // => null


The exclusive range operator (..<) generates a Range where the ending value is one step closer to the beginning value. The less-than symbol can be somewhat unintuitive for descending ranges.

enum E { ONE, TWO, THREE, FOUR }
((E.ONE)..(E.THREE)).toList()  // [ONE, TWO, THREE]
((E.ONE)..<(E.THREE)).toList() // [ONE, TWO]
((E.THREE)..(E.ONE)).toList()  // [THREE, TWO, ONE]
((E.THREE)..<(E.ONE)).toList() // [THREE, TWO]

Indexing a List with a Range only considers from and to values after the above adjustment; the result of Range.toList() is irrelevant, and whether or not it was an exclusive range is not known. Three steps are performed to get the result:

  1. Negative values are normalized to positive values by adding them to List.size()
  2. The result is generated from List.subList( min(from, to), max(from, to) )
  3. If from > to, the result is reversed

In Ruby, a[0...-1] means “From the 0th element up to and excluding the last element,” whereas the ostensibly equivalent construct in Groovy, a[0..<-1], means “From the 0th element to the 0th element.”

a = [0,1,2,3,4]

a[0..-1]             // => [0, 1, 2, 3, 4]
a[0..-2]             // => [0, 1, 2, 3]
a[0..<-1]            // => [0]
a[0..<-2]            // => [0, 1, 2, 3, 4]

a[0..-2]             // => [0, 1, 2, 3]
a[-1..-2]            // => [5, 4]
a[-1..<-2]           // => [5]

Compare to Ruby:

a[0..-1]              # => [0, 1, 2, 3, 4]
a[0..-2]              # => [0, 1, 2, 3]
a[0...-1]             # => [0, 1, 2, 3]
a[0...-2]             # => [0, 1, 2]

a[0..-2]              # => [0, 1, 2, 3]
a[-1..-2]             # => []
a[-1...-2]            # => []

TIP: Fixing author in git history

October 23, 2010 by · Leave a Comment 

I always forget to set up my user info on git on new machines before I check stuff in. It’s pretty easy to fix if there’s nobody else in your repo:

git filter-branch --env-filter "\
  export GIT_AUTHOR_NAME=Dade\ Murphy \ \
         GIT_COMMITTER_NAME=Dade\ Murphy \ "

Source: serverfault

Starcraft vs Monty Hall

May 2, 2010 by · 2 Comments 

If you’re not familiar with the Monty Hall problem, it goes something like this:

There are three doors, and one of them has a prize. You choose one of the doors, and Monty opens one of the others that is not a winner. Now you have the option to stick with your original choice, or switch to the remaining door. It might seem counter-intuitive, but switching doubles your odds of winning.

Some people have attempted to apply the same logic to scouting as Zerg using an overlord and a drone in Starcraft. Stated similarly, the problem goes like this:

There are three starting locations, and one of them has your enemy base. You send a drone to one location, and your overlord gets to one of the other locations and discovers no enemy base. Now you have the option to stick with your original choice, or send your drone to the remaining location.

Sounds the same, but in this case, switching has no effect on your odds of winning. It is the same as the “Monty Fall” or “Ignorant Monty” variant of the Monty Hall problem, where Monty opens a door completely at random rather than one which is a non-winner.

The difference is because in the classic Monty Hall problem, you are initially choosing one door, which gives you 1/3 odds of your first choice being right. If you could choose to switch to both other doors, you’d obviously have a 2/3 chance of winning. In fact, this is exactly what you are doing when you switch, even after one of the doors has been revealed.

In the Starcraft problem, you are choosing two locations to begin with, which gives you a 2/3 chance of being right. If you choose to switch your drone to the remaining location at any point, the overlord still has a 1/3 chance and the drone still has a 1/3 chance. In the Monty Hall problem, you switch from only an unknown door (1/3), to the empty door and an unknown door (2/3). In the Starcraft problem, you switch from both an empty location and an unknown location (2/3), to the same empty location and a different unknown location (2/3).

To demonstrate this visually, I’ve made a simulator in Javascript.

Update (5/3):
I added a “Monty Hall mode” to the simulator, the implementation of which may help make this even clearer. Normally, I choose two random locations from the possible enemy starting points, and send the overlord to the first and the drone to the second. This leads to 3 possibilities with an equal chance of occuring: the overlord finds the base, the drone finds the base, or neither finds the base. Only in the final case, which occurs 1/3 of the time, would it be correct to switch. In “Monty Hall mode,” the overlord is not allowed to find the base – so the 1/3 of the time where the overlord would normally have found the base, is now added to the 1/3 of the time when neither finds the base, making it correct to switch 2/3s of the time.

I also added a “Stupid Overlord mode” which demonstrates that the order is important. If the overlord chooses an empty base first, rather than the drone choosing first, the chances of the drone being correct are (obviously) 1/2.
Read more

Mixing Up Code and Data

May 1, 2010 by · Leave a Comment 

From buffer overflows to cross-site scripting, decades of software security flaws can be traced back to a simple design problem: executable code (or otherwise specially meaningful data), and non-executable, black-box data are intermingled in the same channel. To execute arbitrary code, traditional buffer overflow exploits rely on non-executable data trampling execution state and eventually causing data to be executed as code. Cross-site scripting exploits, and all traditional injection exploits, work when intermediary systems fail to identify the difference between code and data in exactly the same way as some other system.

To prevent these exploits, developers are generally advised to canonicalize input and encode data in output. While this is correct, I think it is important to also understand that it shouldn’t be that way. Developers should have to go an extra mile to cause data to be interpreted as code, not the other way around.
Read more

TIP: Using GDB as an Interactive C Shell

March 26, 2010 by · 1 Comment 

Many programming languages come with some way to run an interactive shell, or REPL (read-eval-print loop). This makes it extremely easy to test little bits of code and understand exactly what they do, and is invaluable when learning a new language or library. For example:

What’s the result of (unsigned int)atoi("4294967295") in C?

Even if you know the answer, how quickly can you prove it? How concisely can you communicate the proof via IM or email? What if it’s a poorly documented third-party library function, and not a standard one?

For quick tasks, you can just use gdb which is probably already present on any system that has gcc. Just fire up gdb on any binary, set a breakpoint on main, and run. When it stops you will be able to call functions and examine their results, and many other common REPL tasks. The binary doesn’t matter much, but you should prefer ones with debugging symbols, and if you want to call functions in a particular library, you should use a binary that is linked to that library.

Example session:

~% gdb ./test
(gdb) break main
Breakpoint 1 at 0x8048452
(gdb) run
Starting program: /home/pcl/sandbox/test
Breakpoint 1, 0x08048452 in main ()
(gdb) set $a = malloc(1234)
(gdb) call sprintf($a, "Hello %d", 12345*12345*12345)
$1 = 15
(gdb) print (char*)$a
$2 = 0x96c6008 "Hello 170287977"
(gdb) print (unsigned int)atoi("-1")
$3 = 4294967295
(gdb) print (unsigned int)atoi("4294967295")
$4 = 2147483647

gdb lets you use arbitrarily-named, untyped convenience variables, as you can see in the example. The only practical difference between print $var = expr, call $var = expr, and set $var = expr seems to be that set does not additionally assign the result to a history variable. Obviously you also have the full debugging facilities of gdb available as well.

It is also possible to do this on stripped binaries with no ‘main’ function, but there are many disadvantages:

~% gdb `which echo`
(gdb) inf files
	Entry point: 0x8048be0	0x08048154 - 0x08048167 is .interp
(gdb) break *0x8048be0
Breakpoint 1 at 0x8048be0

For a fully featured REPL for C, check out c-repl.

Analysis of Gemini Cybernetics CDS

March 24, 2010 by · 57 Comments 

There have been some rumors going around about a new third-party system for Second Life. The system attempts to detect avatars using third-party clients capable of duplicating objects without the creator’s permission, and the rumors are that it uses some kind of QuickTime exploit or other nefarious means to actually examine the contents of your hard drive or otherwise invade your privacy without permission. I decided to take a quick look to see what it’s all about.

The system in question is called GEMINI CDS Ban Relay and is advertised as a simple object which detects avatars entering your sim, and uses “a team of bots with special abilities” to determine if the avatar is “harmful.” If they are, it adds them to an external database, and can optionally ban or teleport them home. Entries in the database are permanent, so if an avatar has been considered harmful once, they are always considered harmful in the future. It claims to use several frequently updated methods to detect “illegitimate” clients.

The most obvious detection method, and the only one I discovered, is a script that triggers as soon as you enter a protected sim and tells your client to load up a special media URL. Using a tool like Wireshark or ngrep, it is trivial to watch the HTTP request.

Read more

Speaking at MBCSC2010

March 17, 2010 by · Leave a Comment 

I will be speaking at the Myrtle Beach Computer Security Conference on April 15th. The title of my presentation is “Practical Web Application Security,” and will partially be a rehash of my post about shared hosting but with more focus on why the little things matter and less on shared environments.

TIP: Make bash tab completion ignore .svn directories

March 16, 2010 by · 7 Comments 

Having to tab through the fifty million otherwise empty “net/mycompany/project/unit/subunit” directories that the Java ecosystem necessitates has consistently driven me crazy because completion stops at each step to let me choose the .svn directory, and I have to look and type the first letter of the directory I actually want to make it continue.

It’s actually really easy to fix this:

export FIGNORE=.svn

$FIGNORE is just a colon-separated list of suffixes to ignore when doing tab completion.

Streaming Nokia N900 Camera to VLC

February 3, 2010 by · 26 Comments 

I recently had need to look at the back of my own head, and using the camera on my phone seemed like the easiest way to do it. I found a guide on the Maemo wiki, but it was for the N800 and I didn’t have the hantro4200 encoder it was trying to use. After learning more than I ever wanted to about gstreamer and sdp files, I came up with a way that works for me.

In my setup, my computer is and the phone is You will have to replace them with your own IP addresses.

Here is the command to start gstreamer on the phone. You will probably want to put it in a script:

gst-launch v4l2camsrc device=/dev/video0 ! \
           dsph264enc ! \
           rtph264pay ! \
           udpsink host= port=5434

If gst-launch is not found, you probably need to install the gstreamer-tools package:

apt-get install gstreamer-tools

To use the camera on the front of the phone, you can change the device to /dev/video1.

Here is the minimal sdp file I was able to use with VLC to get it to play. Using the “Open Network” dialog to try and play an RTP stream did not work.

m=video 5434 RTP/AVP 96
c=IN IP4
a=rtpmap:96 H264/90000

The second line (m=) contains the port, the third (c=) contains the IP address of the phone, and the fourth (a=) specifies the codec.

To use MP4 instead of H264, you can just change h264 to mp4v everywhere. In the SDP file, it should be MP4V-ES, as in: a=rtpmap:96 MP4V-ES/90000. If you get errors in VLC like:

avcodec warning: cannot decode one frame (14922 bytes)

Then add send-config=true to the rtpmp4v part of the gstreamer pipeline, and make sure you start VLC before you start streaming:

gst-launch v4l2camsrc device=/dev/video0 ! \
           dspmp4venc ! \
           rtpmp4vpay send-config=true ! \
           udpsink host= port=5434

For H263, you can try dsph263enc, rtph263pay and H263-1998 or H263-2000, but I couldn’t get it to work.

I don’t know if there’s a way to control the focus, white balance, etc, but I was able to use the flashlight-applet to turn on the camera LEDs while streaming after I downgraded to 0.2-0:

apt-get install flashlight-applet=0.2-0

Three Most Commonly Used Passwords

January 23, 2010 by · Leave a Comment 

Hackers movie screenshot

PHREAK: Alright, what are the three most commonly used passwords?
JOEY: love, secret, and uh, sex. But not in that order, necessarily, right?
CEREAL: Yeah but don’t forget GOD. System operators love to use GOD. It’s that whole male ego thing.

Analyses of various password leaks:

I think it is interesting that as bad as the passwords in Hackers seem, the passwords people actually use are somehow even worse. Where it’s allowed, 123456 always takes the number one spot, usually by a huge margin; in the RockYou leak, 123456 was used 4x more than its closest competitor (12345).¬†When purely numeric password are forbidden, password is the clear winner, and continues to take the number one spot as requirements are added.

Require a capital letter? Password
Number? password1
Both? Password1

The top three I’d try, without knowing the requirements:

  1. 123456
  2. password
  3. password1

    Next Page »