Skip to: Navigation | Content | Sidebar | Footer

Full Archives

SHA-1 and DB Integrity

April 24

Applying cryptographic functions for fun and profit.

I've got two books sitting on my desk, PHP and MySQL for Dynamic Web Sites by Larry Ullman which is absolutely fantastic for bringing yourself up to speed if you already have rudimentary programming skills in other languages (though it could stand for an update), and the PHP Cookbook, an O'Reilly book which (so far) seems like a good reference for getting beyond the basics.

So, any guess on what I've been working with lately? I've been running this site on a pretty basic PHP foundation for a while now, and by basic I mean aside from some PHP includes, passing a couple of variables from page to page, and a slightly more complex spaghetti mess of a comment preview page, there's not much PHP to be found here.

Relational databases, on the other hand, are an entirely new ball of wax. I've worked with people who use them. I've studied the flowcharts. I've formatted the data coming out of them. But I've never had to get my hands dirty before and actually make those connections myself. It's been an interesting learning process as I've put together an application I've needed for a while, which really is the best way to learn. Reading a book and not applying the knowledge immediately is a good way to ensure you don't retain it, I find. Anyway, on to the point of this article, which otherwise probably would have appeared out of left field if not for the above.

This is a particular problem I've run into. Up until recently, all user interaction with the database was limited to adding records. Aside from sloppy code, there were no real security risks to speak of. The new functionality I wanted to build would expose records for editing by the user, which is obviously problematic if the wrong people get access to them.

So the editing interface should be hidden behind a layer of verification. There are a few ways of going about this, a login form being the most obvious. Generating a username/password doesn't make much sense in this context though; the ideal solution would be just a simple text link to get back into the interface and retrieve the data, which requires a unique querystring. Luckily, the data they had previously submitted provides ample doses of uniqueness, so generating one isn't a problem. But this is also where the dilemma comes to light.

Let's assume the unique data in this case is their name. So 'Jane Smith' is told to re-enter the application with a URI like so:

Simple enough. Except guessing a different returnID value isn't exactly difficult, so some records may potentially be exposed to the wrong people. Not good.

So it seemed the most effective way to fix this lay in using a cryptographic hash function like MD5 or SHA-1. (And yes, both have been broken by now, but collisions seem pretty much irrelevant when the hashes exist in a one-to-one relationship with only a few thousand records.) By running the name through the sha1 function:

  $hash = sha1("janesmith");

We get a value that makes guessing pretty much impossible:

But we're not quite done yet. The last step is actually verifying the hash once the user returns, and this poses a choice between two equally not-very-good options. First, we can check the hash against those of each record calculated on the fly:

  $hash = $_GET['returnID'];
  $query = "SELECT name FROM table";
  $result = @mysql_query ($query);

  if ($result) {
   while ($row = mysql_fetch_array($result, MYSQL_NUM)) {
   if ($hash == sha1($row[0])) $successfulReturn = true;

There's probably better syntax for that loop, but you get the idea. Presumably, though, this won't scale very well. I didn't bother testing, but I could easily imagine a few thousand dynamically-calculated SHA-1 hashes bogging down the server in a hurry.

So instead, storing a pre-calculated hash value for each record seems like a good way to ease up the load. Each new record inserted gets a value in the hash column, encoded with the sha1 function of course. Verifying is now as easy as a straight up string compare:

  $hash = $_GET['returnID'];
  $query = "SELECT record_id FROM table WHERE hash='$hash'";
  $result = @mysql_query ($query);

Due to SHA-1 hashes being 40 bytes, there are minor storage space issues, but considering 10,000 records only add up to 4MB, it's not worth obsessing over. It bothers me to have that dead weight for a somewhat minor function, but it's the best I came up with. I'd be curious to compare notes with anyone else who's grappled with similar problems—is SHA-1 overkill? Is there an easier way to do it that I've overlooked?

Permalink › | 66 comments


April 18

Oh good, tabbed palettes are coming back to Flash and Fireworks.

Alright, this one is a little hard to process: Macromedia just got bought by Adobe in a deal worth $3.4 billion in stock. It's hard to say that without saying it again: Adobe just bought Macromedia. (asssuming shareholders and regulators approve etc. etc.)

Adobe Creative Studio MX 2004

Adobe started off making fantastic products for print designers, and has spent the past 7 or 8 years trying to get a foothold on the web. Unsuccessfully, I'd add, as the last time anyone used an Adobe product to generate actual production-ready HTML code would have been some time before the Euro went into wide circulation. Discounting the production work that happens in Photoshop/Illustrator, there are no stand-out Adobe web development tools. LiveMotion? Dead and gone. GoLive? GoLifeSupport. ImageReady? Well, it's been a while since I've personally hit the little button at the bottom of my Photoshop toolbar, but at least that one gets used by other designers from time to time. (Okay, and my GoLive crack was a cheap shot too, but I couldn't resist.)

Macromedia, on the other hand, has been the young-and-hip counterpart, building a product line almost exclusively focused on the web. Dreamweaver, Flash, Contribute, Fireworks, ColdFusion, need I go on? Except in a business sense, they've only been mildly successful, evidenced by the fact that they don't totally own the market they're in.

So now the two join forces. Why? Oh, I think Flash might have something to do with it. Competition over the formats and tools of tomorrow's rich applications is heating up, and Adobe didn't have a foot in the door until today. Sure, they've been big on SVG all along, if perhaps by default, but there are a lot of reasons why SVG is probably not going to be the lingua franca of the future application space. Low implementation rates, a hazy future, and very little developer interest. (Although, interestingly, SVG has a mobile version which is apparently gathering a lot of steam in the handheld space.)

Flash, on the other hand, is a viable option here and now. It has the potential and the developer base to build tomorrow's applications. A couple of questions naturally follow: what will become of Adobe's long-standing commitment to SVG, now that Flash is in the fold? Since SWF is an 'open' format and Macromedia makes money on the player anyway, would there be a business case in transitioning SWF to SVG?

But this is also where the fear, uncertainty and doubt kicks in. If Adobe is gearing up for a battle in the rich application space, what happens if they lose? They're not exactly known for their coding tools at the moment; presumably, key Macromedia teams will be in charge of ensuring the design tools work well with the development tools. From a technical standpoint, a lot remains to be proven. From a competition standpoint, well... as best as I can tell, their main competition is a stable of open source technologies, and Microsoft. That's almost a rigged game, where a likely outcome for Adobe is a thorough trampling by both other interests. Remember how Microsoft sent a rep to promote their latest and greatest to a crowd of Flash developers?

The combined Macromedia and Adobe stable of design software is industry standard; with this buyout, Adobe essentially has a monopoly over the design world. (Quark aside. Very far aside.) So they'll always have that to fall back on. But investors like growth. If the growth stops, or slows, or does anything short of grow, investors react. This particular method of growth is risky, given the competition, and if it fails to materialize, then what? Another buyout? By whom...?

And this is just one particular line of inquiry of many. There are hundreds of other questions which have been asked elsewhere, regarding product lines and feature merging and PDF and and and. Over time, we'll get the answers. Here's hoping this move pays off.

Permalink › | 48 comments

Avalon/XAML First Look

April 14

Familar with Microsoft's new XAML? I wasn't, until recently.

During the first day of the recent FITC conference, Karsten Januszewski (a Microsoft tech evangelist) gave a talk on the forthcoming Avalon. The relevance to the audience was that advanced media and UI features made possible by Avalon will eventually require creators, many of whom Microsoft hoped to reach here. Developers, developers, developers...

My familiarity with the technology up to that point was limited; I had basically thought of it as Quartz (OS X's advanced screen rendering system) for Windows. I'm still not convinced the technology itself is much more than that, actually. Avalon composites many different types of media—text, images, video, and vector shapes. I'm somewhat unclear how detailed the native 3D rendering support is. There was a demo of multiple video sources being mapped to rotating spheres, so there's at least a few basic shapes and textures. My confusion is partially the result of the 'special guests' pulled on stage—a software company that created a 3D -> Flash convertor, which it had adapted for Avalon. Given Flash's lack of native 3D object support, hence the need for this tool, I'm left wondering what that means for Avalon's 3D support.

Quartz vs. Avalon

Without getting too carried away with the specifics though, what seems to be the main difference between Quartz and Avalon is the openness and flexibility. Avalon goes hand-in-hand with XAML, Microsoft's upcoming eXtensible Application Markup Language, which is XML in name if not in spirit. The upshot of which is that controlling the display is as simple as editing a human-readable text file. (Although the Microsoft rep was quick to point out that by no means will this be the only way to do it as current and upcoming design tools adapt to generate XAML automatically.)

That everything in Avalon runs on text files is actually pretty slick. I'll give Microsoft substantial credit for that, it's going to ensure a lot of adoption. Put anything in text and it's pretty much an open format to some degree. But there are two things that detract from what otherwise might have been full points.

The Code

The code itself looks like (well-formed) tag soup from 1997. Whereas the web has seen a shift from presentational markup (in the form of tables, embedded attributes like bgcolor, and the dreaded font tag) to structural markup with a separated presentation layer (CSS), XAML is purely a presentational language. I couldn't see evidence of attention toward semantics, and all the presentational attributes are embedded right in the markup. Januszewski referenced 'a CSS-like syntax', but there's nothing CSS-like about it. It's ugly presentational HTML all over again. A sample snippet:

   <?xml version="1.0"?>
       Width="500" Height="500" Background="White">
         <LinearGradientBrush def:Name="RedGrad" 
           StartPoint="0,0" EndPoint="1,1">
                  <GradientStop Color="#FFFFFF" 
                   Offset="0" />
                  <GradientStop Color="#FF0000" 
                   Offset="0.5" />
                  <GradientStop Color="#000000" 
                   Offset="1" />
              RectangleLeft="0" RectangleTop="0"
              RectangleWidth="500" RectangleHeight="5000"
          <SimpleText Margin="5" FontSize="14">
            An h3, this is not.

It appears to me this is single-purpose code, rather than the layered approached of HTML/CSS/DOM. Can it degrade? How much thinking has Microsoft done about accessibility? What will this code do on a PocketPC or PalmOS cell phone?

Maybe this code makes sense from a desktop computer application development context, but it does not work in today's web development context. The separation of structure and presentation is now a no-brainer, so I'm left wondering why it has been completely ignored here. It's like the past few years of undoing the markup sins of the late 90's haven't happened. Or, they haven't happened at Microsoft. (Other divisions [MSN] seem to get it, to be fair.)

XAML on the Web

Which brings us to the second problem—XAML on the web. Yes, XAML is designed to run inside a browser. It appears a special XAML plug-in or player or something equivalent is necessary, as it was demonstrated running inside of a stock version of IE6 under Windows XP. But Microsoft 'loves the browser' according to Januszewski, and wants it to be painless to select a link on a web page and open a XAML app as a result. Longhorn will presumably feature tight integration of the two.

The past couple of years of IE security problems have raised a lot of awareness about browser lock-in, and the problems that tying your application into a specific browser/operating system can later cause. Like ActiveX, in-browser XAML/Avalon appears to be a method of continuing that trend. Something tells me we won't be seeing an Avalon player for Linux any time soon.

I'd be a whole lot more comfortable with XAML if it were strictly meant as a Windows OS rendering language. Proprietary markup on a proprietary platform is nothing to get worked up over. But the obvious web cross-over leads me to hope we're not going to see a whole new generation of browser/OS-specific web apps. I wonder if Microsoft might be hoping for something different.

If anyone who understands XAML better than I would like to come in and tell me how backwards I have it, I'd love to hear from you, comments are open. I'd rather my first look is all wrong and my worries unfounded (even though I'd like to think I understood the presentation properly).

Permalink › | 51 comments

FITC Roundup

April 13

Some disjointed thoughts about Toronto and Flash in the Can.

I just got back from the better part of a week in Toronto for the annual 'Flash in the Can' conference. The most common reaction I got after mentioning that I'd be speaking at this particular conference was, naturally, why? I very rarely use Flash, and my topic was CSS-related anyway.

As I said while speaking, the best I could come up with was that I got invited as the 'token CSS guy'. There were other non-Flash presentations though, some more creatively-inclined (with obvious relevance) and some more technically inclined (with slightly less obvious synergy). In that context, I think for both myself and the audience, the talk I did actually fit in.

What surprised me was the interest. For my 9am presentation, arguably the worst time slot of any conference, the room was surprisingly full. During Q&A at the end, some very thoughtful and considered questions came up. I could tell that many in the audience were experienced CSS users. Not quite what I was expecting.

Flash in the Can itself was a well-organized, smoothly run conference. I never did get a chance to meet and thank the organizers, so thanks for the invite Shawn.

I alternated my time between seeing a few other presentations and exploring Toronto. I managed to catch most of David Carson's talk and a few minutes of Yugo Nakumura on the last day. It seemed that the presenters I specifically wanted to see were content to show their work, say a few words about their process, and end without having covered anything terribly profound.

The highlight of the talks I saw was James Patterson, a guy I'd never heard of before. When he started running through his work though, I immediately recognized his style. He's done enough TV spots that you may too. While you could never call it pretty, it's uniquely distinct in a grungy, 'flawed beauty' sense, and I found his creation process absolutely fascinating. He's into automatically generating a final result from simple animation components. He starts on a very low, modular level and strings together a sampling from his huge library of previously rendered and animated pieces to create the final result. Quite an eye-opener.

It being my first time in the city (I'm surprisingly non well-travelled in my own country), I made a point of exploring Toronto. The first day or two were extremely disorienting. Almost every flight I've taken over the past three years has landed in the US. Deplaning from a 4.5 hour flight to a city that was still a part of Canada took some adjustment.

Yep, I did the CN Tower. The top observation deck is equivalent to 147 stories high, which (it's claimed) makes it the highest one in the world. There's a glass floor on one level which is about 110 stories high. Talk about a psychological barrier—I'm not afraid of heights, but walking on a clear floor that high up was extremely uncomfortable.

And finally, the people. What I most enjoy about going to conferences is the cool people I get to meet. After my presentation on Sunday morning, I shook hands with a guy who turned out to be Daniel Burka of SilverOrange (and Firefox logo) fame, and his former co-worker Geoffrey (sorry, I never did catch your last name.) Both were great guys with whom I ended up grabbing lunch and wandering the city.

And we did a dinner with the group. As bad with names as I am, I believe those present were Joe, Daniel, Suzanne, Mike, Brice, Joanna, Craig, and no doubt one or two others I'm missing. It was good to hear about a group across the country doing what I wish we had going here in Vancouver. It might be time to get moving on that...

Coming tomorrow: a discussion of Microsoft's FITC presentation of Avalon (their new rendering system) and XAML (the code that goes along with it).

Permalink › | 13 comments

Inside the World's Biggest Bookstore | April 9

Try walking into a bookstore you've never seen in a city you've never been to and find something you wrote sitting on the shelf sometime.

View on Flickr › | 2 comments

Google Maps and Accountability

April 7

The ease of use of Google Maps' new satellite imagery integration is going to have some interesting implications.

A few days ago, Google integrated its Keyhole technology into Google Maps, allowing an easy one-click switch between map data and satellite imagery.

It's a mind-blowing use of technology. If you've played with the full version of Keyhole before, the novelty is a bit dampened (especially since the web data is quite a bit lower in resolution), but the ability to toggle back and forth between map and satellite views is insanely useful for establishing a sense of scale.

Interesting things are happening already. Matt Haughey has been exploring and annotating what he's finding. Jeff Veen thinks it's a fulfillment of an Orwellian prophesy, by the people instead of by the government. Privacy concerns for the individual? Well, there are bound to be those. Read the comments on Jeff's "Google is Watching" post for intelligent rebuttals—it's not even close to real time data for one, and Flickr has much more potential as a surveillance tool anyway.

But corporate and environmental whistle-blowing? Let's kick that off right in my own backyard. The province of British Columbia has a huge and thriving forestry industry. There are a lot of things one could say about trade tariffs and exporting practices and pine beetles and other business concerns, as they've been in the media lately. I have mixed feelings about that industry in general, as it is important to the local economy, and I've done my share of work for companies that earn their money from cutting down trees.

But that's as far as I'll go to defend it. A picture is worth a thousand words and all that, so here's a 4000 word essay on what the forestry industry is doing in British Columbia, as of whatever the date was when the satellite snapped these. Click through any image to get to the Google Maps bookmark of the same, it's worth it to move around a bit and get a sheer sense of scale (and see how much more there is to this than what I've depicted in these 4 shots.)

Close-up view
City of Kamloops
Wider View

What's it like on the ground in one of these clear cut areas? Well, if you're lucky and the operators of the cut have actually replanted anything, it's generally a young glade of small but healthy trees.

If they haven't, it's ugly. The bleak ground is covered with gnarled and torn wood left behind to rot in the elements, and muddy where the root systems that used to keep the soil together have died. It's disgustingly barren, while at the same time being impassable due to the amount of dead vegetation littering the destroyed ecosystem. I wish I had some photos of that, perhaps I'll see what I can dig up.

Permalink › | 42 comments