Nicholas Weaver's Random Thoughts

A Protocol for Visiting China (or DEFCON)

2009-07-29T08:06:00.001-07:00

The following is my computer protocol for visiting a hostile environment. I actually designed it under the threat model of "What if I needed to visit China", which requires facing two nation-state adversaries (the US and Chinese government) which may have legal access to the computer, but I use it for going to DEFCON.

It may actually be overkill for DEFCON, but as they say "There is kill, and there is no kill, there is no such thing as overkill". I wanted something I knew could work.

The philosophy is twofold, the first is system hardening, while the second is constraining the damage a compromise could do.

I begin with a clean OS install on a newly formatted hard drive. The system is brough fully up-to-date and necessary tools are installed (Firefox, Bro, Click, ipsumdump, Tex, etc) that I will need during my trip. Plastic MacBooks are especially nice, as the hard drive is trivial to change.

I then segregate data. I create a new account on a server I have access to. This account has a new password, and is accessed through a new SSH private key. I create a version control archive on this account which I can also access from my normal account(s), and use this to store the entire working set I will need during the trip, but no more.

Finally, I set up my web browser. I use NoScript, disable flash, disable Java, and tunnel all traffic through SSH. (I use both browser hardening and a tunnel because its easy to screw up and have traffic escape a tunnel, eg, by forgetting to set Firefox to also tunnel DNS through SSH).

This works not because what is present, but what is absent. I do not have access to my mail accounts, normal public keys, or full working set. Not only do I harden my system, but I explicitly limit the working set so that a compromise minimizes the damage. If I need email access during the trip, I will set up a new Gmail account and forward my mail to the new account.

And once I do that, no worries! I may be on a hostile network, but I've taken steps to minimize my vulnerability surface. But I know I'm not perfect, and who knows what zero-days are lurking in my computer. Thus I've limited the potential damage from a compromise: you can't compromise data that doesn't exist.

On Lauren Weinstein...

2008-09-04T12:21:00.001-07:00

I was, for a long time, a participant on Lauren Weinstein's "NNSquad" mailing list. There are many important issues in traffic shaping, traffic management, and other related topics. But I had to conclude, reluctantly, that you can't deal with him. His views are those of a zealot, unwilling to compromise, and seems intent on forcing his view of "neutrality".

The two last straws were his censorship policy and his belief that a high usage cap (250GB, what Comcast is doing) is somehow significantly anticompetitive.

His reaction to Comcast's cap mystifies me. Its almost a total victory for his view: Its transparent, its neutral, its not anticompetitive (250GB/month is >7 hours/day of 720p HDTV video delivered over the net.), and because the response is to first warn than terminate customers over the cap, it CAN'T be used in an anticompetitive manner.

And for a list which is supposed to be about "open exchange of ideas", he's an incredibly harsh censor.

This was my "goodbye to the list" message, which he did not print, of course.

He notes that his part of the conversation was for "private consumption". Yet nothing he said in the private discussion is anything different than what he has said publically on multiple occasions.

It is his sandbox. He can say what he wants, and ignore dissenting views. But by the same token, everyone else should be aware that this is how he operates.

To respect any copyright he might possibly claim, I've excluded his sections, replacing them with paraphrases.

Lauren, folks.

I have to conclude the following: This project will be a failure.
Period. Because even if you "succeed", success will lead to
usage-based pricing. You have proven that you will accept nothing
else.

And, Lauren, you really have practiced heavy censorship, not of
personal attacks but of technical discussion.

You have proven unwilling to acknowledge, respond, or publish the
following, which was an on topic, technical discussion of the issues.

How does the following not perfectly mesh with your stated moderation
policies? Yet it seems to have gotten dropped down the memory hole!
You don't want cooperation. You don't want open discussion.

Thus this is "So long": This project will fail, and the mailing list,
due to Lauren's policy of squelching open discussion which doesn't
agree with his preconceived notions, has already failed.

On Tue, Sep 2, 2008 at 9:46 AM, Nick Weaver wrote:
> Replying on-list.
>
> On Sat, Aug 30, 2008 at 6:30 PM, Lauren Weinstein wrote:
>>
>>> Well, Google's low bandwidth, so it doesn't matter.
>>

{Here lauren insists that google is major bandwidth in the aggregate for both user queries and spidering}

> Google's bandwidth in the AGGREGATE is trivial from the end-customer
> perspective compared with the HD video services which you argue about.
>
> Even google apps is light: Start up google docs, start up a new
> spreadsheet. Thats less than 1 MB transfered (there is other activity
> in my mini-tracelet, so 1 MB is an upper bound). And most of that
> should get cached the second time around.
>
> Or 20 seconds of 400 kbps video. Thats IT. Google (sans video) is
> NOT high bandwidth, even when dealing with a lot of customers,
> compared to video applications.
>
> And even from the webserver side, its not THAT bad.
>
>
>>> But cuil is a
>>> counterexample, where it is not only going up against the ISP behavior
>>> that is the threat, but also a huge existing competitor: google!

{Lauren's comment: Cuil is not proven to be viable.}

> Yet you seem to insist that the network uncertanty should prevent it
> from even being funded?!?
>
> "Would Google be successfully funded if it were trying to get off the
> ground in today's Internet environment? A question to ponder."
>
> Answer: YES, because Google competitors, and much higher bandwidth
> services, are both being funded!
>
>>> But lets look at high-bandwidth new ventures, which do directly
>>> compete with ISP offered services. Like, oh, Youtube, Hulu, etc.
>>> Since Hulu wasn't even launched as a joint venture until 2007, well, I
>>> think this is another data point against your hypothesis.

{Lauren calls youtube relatively low bandwidth}

> Hu? You say that "Google is high bandwidth" but "youtube is not"?!?
>
> Yet your claims that 250 GB cap is somehow anticompetitive relies on
> video services which are an order-of-magnitude larger than YouTube in
> bps.
>
> You can't have it both ways. Google, even google apps, are
> lightweights in comparison.
>
> And I use Hulu as a DATA RATE example, and an example of a company
> today which is really stressing the limits of Internet video delivery,
> complete with full Akamization.
>
>>> As for 250 GB, it was not an arbitrary choice, even if it seems so to
>>> you. Rather, its a round number approximation of the 1% heavy-tail
>>> today.

{Lauren asks me where I get this, and asks how I justify Time Warner's proposed 50GB cap}

> On comcast, speculation based on communication with multiple
> individuals in various ISPs, and actually believing comcast's
> statement that <1% would be affected, based on experience with network
> operations. 250 GB is a lot of data.
>
> There is a reason why, even I, as a researcher with an underutilized
> 100 Mbps pipe to the Berkeley campus, with its >1 Gbps pipe out to the
> rest of the Internet, if I had to transfer >250 GB in a research
> project, I'd use Fedex.
>
>
> And have you been listening?
>
> I DON'T justify TW's 50GB cap. That is exactly the cap level you want
> if you want to be anticompetitive: it keeps all the websurfers and
> casuals happy, but it kills any attempt to do a lot of video over the
> net.
>
>
> That you treat the two the same is the heart of your problem: If you
> and your ilk are going to claim that any cap that could be potentilly,
> possably, maby anticompetitive in the future is just as evil as one
> which is anticompetitive today, why should any ISP listen to you?!?
>
> The ISPs are your frenemies, not your enemies. They are delivering an
> incredible service at incredibly low cost. You should work with them.
> Yes, you need to watch them like a hawk, but they are also rational
> actors and can be worked with.
>
> But your reaction to a reasonable cap, by being effectively the same
> as your reaction to unreasonable caps, has made it clear that there is
> no satisfying your position.
>
>
>>> Be thankful the model isn't "Over the limit? Throttle all traffic to
>>> 100 Kbps" instead, because THAT model is far less cost for Comcast, so
>>> there would be an incentive to reduce the threhhold to affect more
>>> users. But in the model of terminate if over limit, if ever more than
>>> a percent or so are affected, Comcast becomes the one with the serious
>>> problem, not Comcast's customers.

{Lauren says he prefers throttling vs cutoff models}

>
> Riddle me this, then. Which is more anticompetitive at preventing
> video over the net:
>
> a) A 250 GB/month cap where, if you go over, the first time you get
> called and the second time you get cut off?
>
> b) A 50 GB/month cap where, if you go over, you get throttled down to
> 400 Kbps?
>
>
> Lets see, the first is high cost to the customer if triggered, but
> also very high cost to the ISP, and only affects a trivial number of
> users today and even tomorrow, assuming 2.5 Mbps 720p video.
>
> The second is low cost to the ISP, mid cost to the customer, but
> pretty much guarentees that video-over-the-net can't be used as a
> significant form of entertainment.
>
> Cutting off users is an activity that a business can only take if they
> really are at levels that are abusive to the network. You WANT the
> reaction to be fully-cutting-off-the user: it greatly increases the
> cost to the ISP.
>
> User-terminating caps are far more neutral than bandwidth-throttling
> caps, because of the cost to the ISP means that a user-terminating cap
> can only be deployed in really extreme cases, especially when one
> considers the multi-service aspects.
>
>
>>> If Comcast stated that "We will increase the threshhold as demand
>>> grows so less than 1% of customers would ever be affected", would THAT
>>> be satisfactory?

{Lauren basically insists he won't believe it}

> What would it take? Auditors? If it was an audited statement, would
> you accept it then?
>
> Because I don't see how you can convince anyone that a cap affecting
> <1% of the users would have a significant anticompetitive effect. It
> EXACTLY meets your criteria below.
>
> Will you accept the following statement:
>
> IF a bandwidth cap affects fewer than 1% of the customers, it is not
> significantly anticompetitive.
>
> Yes or no.
>
>>> Can ANY cap be satisfactory to you?

{Lauren requires that any cap be justified, and complains that we don't have visibility into the networks in question.}

> A lot of the capabilities are directly reversable from the technology.
> Its all Gige with occasional 10 GigE from the DOCSIS hub, its all
> DOCSIS 2 with some 3 rolling out (but the DOCSIS 3 rollout only
> affects downstream, not upstream).
>
> The physics and all are well known, and if you wanted the details of
> just how many customers and how much frequency range is on a user's
> CMTS, look at the DEFCON work on sniffing cable modems.
>
> DOCSIS is a broadcast medium. I suspect that even with encryption
> turned on, you should be able to get all the information you want on
> the actual internals of the cable company's residential networks.
>
>
> Given a user at the 250 GB/month, 8 hr/day duty cycle, that user is at
> 2 Mbps. Since a DOCSIS channel is only ~40 Mbps, that user is tying
> up 5% of an entire cable channel for the whole neighborhood. Thats a
> big cost right there.
>
> Likewise, price out COMMITTED data: price out a T1. Thats a good $100+/Mbps.
>
> Lets assume Comcast's committed rate is 1/5th of that, say $20/Mbps.
> A user at 250 GB is going to use ~1 Mbps continuous, which means at
> MINIMUM, assuming they were at a continual low rate, the user costs
> $20/month. Since in reality users are bursty, AND somewhat diurnally
> synchronized, a 250 GB/month user could easily cost the ISP $60, 80,
> 100+ in transit cost alone.
>
>
> You don't need to trust the ISP's statements to know that a 250
> GB/month user is a severe moneylosing proposition, you just need to do
> a little math.
>
>>> 1) No traffic shaping: best effort only and let the end-points fight it out.

{Lauren says "Voluntary" traffic shaping, well defined, and doesn't skew costs.}

>
> There is no such thing as "voluntary" traffic shaping between users.
>
> And I suspect there really is no satisfying you on traffic shaping either.
>
> EG: a policy like this: "The network enforces fairness such that
> viewed over a time average of X minutes, all users have an equal share
> of bandwidth when congestion occurs".
>
> Now if you talk to Comcast's engineers, and watch their presentations
> at IETF meetings, you'll understand that what they are doing with
> their farness solution is trying to approximate that with simple
> measurement and two QOS bins, so they don't need to buy new equipment.
>
> Yet look at the reaction in this forum to it!
>
>>> 2) flat-rate billing, [1]

{Lauren says there may be cases where usage based pricing is OK}

>
> I am willing to bet that any usage-based pricing scheme that a company
> would deploy would kill your "HD over IP" dreams.
>
> Do you care to take that bet?
>
>>> 3) A significant committed information rate.

{Lauren asks what do I mean by significant}

> You and others seem to subscribe to the "bandwidth is a scarcity"
> arguments, and the "I bought a 16 Mbps download line, I should get a
> good fraction of that", which implies a huge committed information
> rate.
>
> For "significant", you probably mean at least 1 Mbps. Do you want
> your ISP service to cost $100/month more just to give you that?
>
>>> 4) All other services offered by the ISP should be treated as
>>> bandwidth-equivelent with the internet service for 1,2 and 3.

{Lauren notes this is case-by-case}

> If you can't at least approximate these costs with a
> back-of-the-envelope however, you are doing something wrong.
>
>>> [1] And of these, #2 is the greatest threat. Because if you accept
>>> usage-based pricing, that will kill off your future "true HD is 10
>>> Mbps encoding" services faster than you can say "$.20/GB becomes $1/hr
>>> for transport. Have you considered US Mail?"

{Lauren claims UPS is an inappropriate example when comparing data-delivery business models}

>
> It is EXACTLY appropriate, because USPS is the competition for ANY
> "data overnight" video service.
>
> The USPS can get you an incredible amount of data overnight. Lets
> see, a BluRay disk is 50 GB. Thats 4.5 Mbps, and a cost/GB of roughly
> $.02/GB.
>
> If you want "data now", even at just $.20/GB, that is $1/hr for the
> movie, period, with transcoding. Or $10 for a full BluRay disk. Have
> a nice day.
>
> With charges more likely to be on the order of $1/GB, what do you
> think that would do?
>
>
> Remember Tanenbaum's famous maxim: "Never underestimate the bandwidth
> of a station wagon full of mag-tape".
>
>
> This is also why I don't believe in P2P for video content delivery:
> For "data now", it adds bandwidth.
>
> For "data overnight", where it would be friendlier than TCP, then it
> competes with US Mail and US mail's incredibly low cost/bit.
>
>>> [2] And for all the talk about the ISP being an evil monopoly, its
>>> really an evil DUopoly, where the ISP service is often used as a
>>> competitive lever across all services. If there was a huge profit to
>>> still be made in being an ISP, where are the metro-area WISPs? The
>>> third party DSL-based ISPs? They died in the marketplace due to
>>> competitive pressures: there is not much profit margin in being an
>>> ISP.

{Lauren states that the third party ISPs largely died because of the regulatory environment.}

> Start your own. Quit whining and start your own ISP.
>
> The legislative environment has almost no effect on point-to-point
> WISPs. All you need is a tall antenna someplace. A minor headache
> with the local zoning board.
>
> And you can still get DSL lines from the incumbent telco (I do for my
> home service) with layer 3 provided by a third party. That system
> still seems to be working just fine.
>
> I suspect for all the complaints about regulation keeping that duopoly
> intact, the bigger problem is just that the cable/telcos view the ISP
> service as something of a loss-leader: voice and video (either through
> new line or sattelite if you are a telco) is far more profitable, but
> IP service can get people to switch.
>

HTTP is Hazardous to Your Health

2008-05-19T14:01:00.002-07:00

The following is not original, but simply a summary of widely known information.

It has been known for decades that plaintext protocols, such as HyperText Transfer Protocol (http) are vulnerable to man-in-the-middle attacks. Yet we are now at the point where there is simply too many ways to man-in-the-middle the web browser, and too much lovely mayhem that can be constructed, for this to be tolerable. We MUST demand that web-sites shift to HTTPS for everything, and ship web browsers that disable http altogether.

How to Man in the Middle: There are simply far too many ways to act as a man in the middle to a web browser. These include maskerading as any access point requested by a system (Karma), ARP cache poisioning (arpiframe), DNS cache poisoning, WiFi packet injection ( airpwn), or simply an ISP attempting to monetize the network (advertisement injection,Phorm). If an adversary can eavesdrop on our HTTP sessions, they can act as a man-in-the-middle.

The problem arises from all the malicious fun that can be done by a man-in-the-middle. This can include:

Cookie Pillaging: By having the web browser transparently redirect through a long list of sites, the browser will transmit EVERY non-secure (not-SSL-only) cookie to the eavesdropper. Which means the eavesdropper can read! your! gmail!! and other such lovely mayhem, as many sites which allow SSL access don't actually set the cookies properly to mandate SSL access, which means from the viewpoint of an active attacker, SSL does no good at protecting the site!
Autocomplete Pillaging: Instead of just redirecting through a long list of sites, include hidden forms and javascript to capture all the autocomplete information present in the browser. A technique developed by H.D. Moore.
SMB Reflection: IE will happily open an SMB share when given the proper URL, which can be the attacker's share on the local network. The attacker can use this for the SMB reflection attack (at least on older systems), allowing the attacker on many systems to read and write the user's directory if file sharing is enabled, or to relay authorization credentials to a third party file server. Its unclear how well this can still work, but its at least worth trying.
Worms: Take the IE 0-day exploit-du-jour and make a worm that uses packet injection/AP spoofing to spread to all other systems on the local wireless network. For extra credit, release such a worm at JFK airport and include a phone-home visit to the CDC website giving the CDC a nice model for how an influenza-of-doom would spread. (Heathrow may be slightly better, but it is far cheaper to have your worm fly to heathrow in your place. Also, the spread rates in the Usenix paper are probably conservative, because they don't model effects like an infected notebook carrier doing work in a taxi)
Drive traffic to your blog:. Gotta have a proof of concept to get people's attention! Note that it only took a couple of hours to hook up a fragile but nontheless working demo. The attack would have been much more effective if I actually played games with wireless transmission power.

So what is to be done? Simple. NO MORE HTTP! Everything, and I mean EVERYTHING should be through HTTPS/SSL. The security community managed to kill off Telnet. We need to do the same to HTTP (oh, and non-secure DNS, too.)

Japanese are going to do "graph takedown"...

2008-03-25T11:11:00.000-07:00

Geore Ou reports that the Japanese ISPs are going to start doing something similar to what I noted in january, albeit instead of just attacking the graphs of communication, simply warning and then disconnecting users.

(typo fixed, grr)

A security thought: AT&T Copyright Fighting

2008-01-27T08:30:00.001-08:00

The following is just my own opinion and speculation, to a hypothetical question: If I was AT&T, why and how would I implement the AT&T plan to enforce copyright on user traffic. (Note, this post is an extension of my slashdot comment on that thread, and basically describes a "DMCA Takedown on the Network Layer" style of response.)

I also believe this would be a significant problem if implemented. I'm a believer that general network neutrality is a mostly good thing. But when a company seriously proposes filtering, I believe we should attempt to determine what shape such filtering would take, and how it could maximize the stated objectives while minimizing collateral damage. This also gives those opposed to filtering a leg up on attempting to counter it.

To begin with, AT&T probably has a huge incentive to block pirated traffic. Time-Warner cable supposedly has 50% of the bandwidth used by 5% of the users. Who wants to bet that of this bandwidth, it is almost all pirated material and/or pornography? As an ISP, wouldn't you want to remove 1/3rd of your traffic? Especially if its customers that can't really complain about it?

The strength of piracy on the Internet is the ease of getting the pirated material,and the ease of distribution. Thus pirated material must be easy to find if it is to be a substantial portion of traffic and to have a significant economic impact.

So all the MPAA has to do is find the easy-to-find content, and do something about it. Currently, they've tried playing Whak-A-Mole on the Torrent tracking servers, but this has been a losing game, as these servers have already fled to "Countries of convenience", where they are difficult for the MPAA to sue off the network.

But rather than playing Whak-A-Mole on Torrent tracker servers (which are largely offshore), with ISP cooperation from AT&T it becomes possible to play Whak-A-Mole on the torrents themselves. Such a system would benefit both the content owners and the ISPs.

All that is necessary is that the MPAA or their contractor automatically spiders for torrents. When it finds torrents, it connects to each torrent with manipulated clients. The client would first transfer enough content to verify copyright, and then attempt to map the participants in the Torrent.

Now the MPAA has a "map" of the participants, a graph of all clients of a particular stream. Simply send this as an automated message to the ISP saying "This current graph is bad, block it". All the ISP has to do is put in a set of short lived (10 minute) router ACLs which block all pairs that cross its network, killing all traffic for that torrent on the ISP's network. By continuing to spider the Torrent, the MPAA can find new users as they are added and dropped, updating the map to the ISP in near-real-time.

This would be a powerful system, and the likely solution AT&T will use if they carry through on their plans to enforce copyright:

This requires no wiretapping. Instead, it relies solely on public information: the torrent servers and being able to contact participants in order to map those fetching an individual file. BitTorrent encryption would have no impact on this scheme.
It can be completely automated, both for the MPAA and AT&T
It also minimizes collateral damage, since only participants in an individual torrent can't communicate with each other when a Torrent is blocked. If the MPAA actually spiders the torrent (rather then trusting information from the trackers), there should be no false edges in the graph. The only collateral damage is if a pair of systems is also performing legitimate communication at the same time they are participating in the Torrent, something the ISP probably considers acceptable.
Any real collateral damage (incorrectly blocking content) AT&T can say is the fault of the MPAA.
It should be robust in the arms race: if the pirated material is open and distributed in a P2P manner, the MPAA's spiders should be able to track it. (Remember, even if CAPTCHAs are used to protect trackers or aspects of the systems, solving a CAPTCHA only costs $.01).
And its inexpensive. All AT&T has to do is deploy a small program to set and release a bunch of router ACLs, and thats it. AT&T can even keep the number of ACLs reasonably low, because they expire quickly and only need to be partially effective. No new hardware is required and everything can be fully automated. All the real costs (of spidering the Torrents, content identification, affirming that it is actually a copyright violation, and constructing the graphs) is placed on the MPAA or their contractor.

Likewise, (IANAL) AT&T can possibly avoid most liability. They aren't doing any wiretapping, nor even making a decision about which traffic to block.

Finally, AT&T has a huge number of reasons to deploy such a system:

It keeps the content providers happy for when they are negotiating their compete-with-iTunes/Netflix video on demand and cable TV services.
It keeps the content providers from pushing through very draconian legislation, or at least draconian legislation you aren't happy with. (It can F-up your competitors, but thats just a bonus)
And it drops their bandwidth bills by 30-50% by eliminating a large amount of deliberately-noncacheable (both politically and because of bittorrent encryption) traffic.

This won't stop closed-world pirates, those with a significant entry and secrecy, but those are far less significant. Closed-world pirates are much lower bandwidth for the ISP, because its far more difficult for pirates to get the content. But it should be able to shut down Bittorrent for open-world piracy, without blocking legitimate BitTorrent. It also won't stop child porn, although AT&T would probably claim that it does.

This was speculation. I have no evidence that this is what AT&T is planning. But given the huge expense (deep packet inspection), legal implications (wiretapping, false positives) and limitations (cryptography), I find it doubtful that AT&T really wants to detect copyrighted material directly. Performing deep packet inspection at line rates, especially to match a large database of copyrighted material, is hugely expensive, and would fail in the presence of encrypted Torrents and SSL-equipped Torrent search servers.

Thus I'm almost certain that if AT&T truely wishes to carry forward with its copyright-enforcement plants, the system will be similar to the one I've described.

Detecting this if they do deploy copyright enforcement would be possible, by participating in torrents (to generate the block) and then checking how that affects connectivity. If AT&T blocks Torrents but other TCP connectivity in those port ranges remains between two hosts, they aren't using only the speculated system, instead they would have to be directly inspecting the traffic between the hosts to determine that an individual flow is participating, information which can only be obtained by directly monitoring communication between the two hosts.

EDIT/addition: Richard Bennet has also discussed this technique at the Network Neutrality forum on 1/26/2008 (Slides at Richard Bennet's web site, on how easy it is to find pirated materials and participating peers to tell the ISP what to block).

He also brings up the important question: "Is there any reason that such an automated system should not be used, or does Net Neutrality now connote a license to steal?" This is a tough argument to counter.

The ongoing discussion can be viewed at The NNSquad Mailing List archive.

EDIT/addition #2: Delayed release of keys (distribute then release keys, as Richard Clayton pointed out) would slow down any spider, but also slows down users from getting content. The spider could still block all users after the key is released, and as people couldn't tell what they are downloading BEFORE the key is released, the MPAA could produce a large number of poisoned (false data) torrents during this window.

Comment spam is worth real money...

2007-12-01T14:58:00.005-08:00

(Note: Links are deliberately not clickable, we don't want to give the Spammers pagerank)

Blogger has a pretty significant amount of protection against comment spam. They have to, because comment spam degrades the blog ecosystem. On this personal blog, I've just gotten comment spam like this:

I have to say that I love this article. I have searched for many weeks to find an article about this topic. This blog has been so simple and has a lot more features than other blog articles. The layout and design is great. I will continue to come back here for every articles. Thanks.........

Eva Maryam (a link to kitchencabinets-online.SPAMblogspot.com)

www.jetblue-airlines.SPAMblogspot.com (another clickable link)

Both blogs are simply full of automated-feed content from www.SPAMgetmyarticles.com, a company which gives free articles for posting on websites, probably mostly spamblogs, to make them seem legitimate. And the rest of the blog is a pile of Google adds.

It cost the spammer roughly $.01 to post that single bit of comment spam!

I have "Word Verification", aka the comment CAPTCHA, turned on. This means the spammer either had to invest a lot of effort in building or buying an automated tool, did it manually herself, or outsourced the CAPTCHA solving to a human, such as the Amazon Mechanical Turk, a porn-for-CAPTCHA service, or a Chinese Turing farm. But in any case, all the alternatives effectively cost real money. A CAPTCHA can protect something valued at less that <$.01 or so, but anytime the value is >$.01, CAPTCHAs are useless because you can always just hire people.

I should consider it a complement that the spammer would spend $.01 to post an advertisement on my inconsequential blog. And thus I can see how the spam blogs make money: there are a lot of adds spewed out on that page and it costs nothing to set up. Just a single click might make the spam-blogger $.10 or $1.00 or even more.

And they make enough money to make solving CAPTCHAs worth it, which means blog spam is far more valuable than email spam. An interesting result, and not good for the viability of blogs when comment spam on a random, nearly unread blog is actually worth money to the spammers.

The best counter is probably to attack the add-blogs themselves. All of the content in the spam blog itself doesn't cost the spammer, but by not paying for hosting the Spam blog, they are vulnerable. If Google actually responds to my flagging of the spam blogs which post comment spam, this would disrupt the spam-blog ecology.

But we will see in the future whether this happens, whether Google decides it benefits more from add impressions through spam blogs or is hurt due to the disruption of the blog comments system. I hope for the latter, as manually removing such spam costs me more that $.01 in my time.

The interesting thing is Get My Article's business model. Its free to use the content, but submitting the content requires paying $20/month, and requires actually creating semi-real content! So why are people paying good money to have their legitimate text articles (albeit complete with links) on people's spam-blogs?

Hofmann's Crash

2007-07-24T07:44:00.000-07:00

I was at MotoGP racing this weekend, having fun with my camera. One of the photos I captured was Alex Hofmann's crash during free practice in MotoGP, when he was T-boned by Sylvain Guintoli on the enterance to the corkscrew.

I was using a rental lense on my Canon XTi, with a deliberately long shutter speed (1/200) to increase the sensation of speed and depth of field.

The full sequence is here.

I've released theses photos under a BSD-style (aka Creative Commons style) license. Since I'm just an amateur having fun, it is better to just get my name out there, and to maximize the number of people who go "Hey, thats a cool photo".

How to (and how not to) run an airline

2007-07-11T13:34:00.001-07:00

For those who have yet to experience the joy of East Coast air travel, there is one bane beyond all others: East Coast thunderstorms. During the late afternoon and evening, masses of thunderstorms often form, blocking airports and flight paths from Boston to Washington. These storms often create "creeping delays", where all Air Traffic Control can tell the pilots sitting on the ground is "ask again in half an hour", because it could be 15 minutes and it could be 4 hours before the planes can fly again.

In two trips within three weeks, I got to experience this first hand.

The most recent, from Washington DC home, was on JetBlue. With a tight travel budget and DC's outrageous hotel costs, an extra night and a morning flight was not in the cards. So on the evening of June 28th, I arrived for the 9:20 PM flight from Dullis to Oakland, Jet Blue flight 321. Sometime around 6pm, the airport basically was shut down: thunderstorms were blocking all routes east and north, and a storm was heading directly to the airport itself.

During this time, other airlines still boarded people and shoved planes onto the tarmac. JetBlue did not. The counter personnel said "we don't want to board the planes until we know you can take off, its more comfortable sitting here". An airline than learned its lesson the hard way.

More important, they communicated with the passengers. Every half hour or so, the counter staff would check in with a pilot on a plane and get an update from Air Traffic Control.

One pilot (the pilot for my flight) stayed at the counter and helped out: explaining to people the cause of the delay, looking up flight status on his smartphone, showing the weather radar to people, assuring us that he was NOT going to cancel the flight to Oakland, and even detailing the tricks he was going to pull to try to get us out as promptly as possible, a scheme which required shanghaing off-duty and over-houred flight attendants to board us 15 minutes before our scheduled cabin crew was due to arrive from a connecting flight.

One of the counter staff even unloaded a few drink and snack carts from the plane, with a "I know this won't make you feel better, but it makes me feel better, so help yourself". The good customer service even continued onboard, with the pilot unlocking the pay-per-view movies.

So although the flight was almost three hours delayed leaving (but only slightly more than two hours arriving, the Pilot put the pedal to the metal), and other flights suffered even longer delays, the process went as smoothly as could be expected.

About the only thing which would have improved the situation would be a weather and/or weather + air traffic display in the lounge, so the customers could see for themselves the airborn mess.

This was in sharp contrast to United flight 19, on June 8th. A similar evening flight, from JFK to San Francisco. Weather was moving in, and any pilot worth his salt would have seen the impossibility of getting off the ground. Nevertheless, we boarded the plane on time.

It turned out June 8th was going to be a ClusterF*** of a flying day out of JFK. Earlier in the day, Air Traffic Control on the east coast suffered a major computer crash. One of the two taxiways at JFK was closed for construction. The East Coast Thunderstorms made their appearance. And an emergency landing on one of the other runways.

But the United pilot told us nothing, simply moved the plane to the taxiway and parked it on the side. No updates, no reports.

The only reason I knew about the weather issues (and resulting routing issues), the emergency landing on the other runway, and most of the other problems was because the pilot did not turn off the ATC channel on the entertainment system, so I listened away to 'xxx, switch to controller C, wait for him to contact you, it will be a while' and 'all emergency vehicles roll to runway Y'.

Even the cabin crew didn't know about what was going on, relying on me to relay information to them! After an hour or so, they distributed cups of water but provided no other cabin service while we were sitting on a taxiway with the engines off.

Four hours later, we were finally in the air. Again, I knew about departure information long before the cabin crew was informed, let alone the passangers. Even during the ascent, the frustrating lack of communication continued, with the pilot detailing to ATC the "moderate" turbulence we were passing through but saying almost nothing to us poor souls along for the rather bumpy ride.

So thus ends the simple lesson in how to, and how not to, run an airline.

iPhone Redux and the Left Turning Porsche...

2007-07-05T11:50:00.000-07:00

OK. I still don't like the lockin policy. I find it horribly objectionable.

But I got a chance to play with an iPhone this weekend, a good 10 minutes of lustworthy exploration.

Yeah, the edge network is sucky, but it is sufficient for a lot of work.

Yeah, the lockin policy is repulsively crippling.

But the thing is so well done, so well put together, so easy to use, with all the little touches, that if I didn't have 7 months to go on my cellphone contract, I'd go out and buy one today.

I'll still probably wait (I don't want to spend an extra $150 bucks to get out of my cellphone contract), but the iPhone looks seriously worth it.

iPhone Lockdown and Intent-Based Pricing

2007-06-21T14:05:00.000-07:00

There are several applications I'd want to run or port on an iPhone. This includes a full ssh environment, subversion version control, and some custom scripts using ImageMagic which would allow me to process, manipulate, and upload photographs using my digital camera (assuming you could adapt the iPod port to a camera or compact flash card) : all tasks I perform on my Mac laptop but which would greatly benefit from the greater portability of an iPhone.

Yet Apple and AT&T's lockdown policy, only Apple authorized applications can run on the iPhone, means I will be unable to use the iPhone to its potential. I understand the reasons why Apple and AT&T want this property: they want to limit applications which can run because they wish to bill for service based on intent.

At $10 for 1500 SMS message at 1 kB/message, SMS messages are worth roughly 1.2 Mb/$. With voice (beyond the first 500 minutes) at roughly $.05/minute and approximately 8 kbps, vocie is roughly 10 Mb/$. Finally, at "unlimited" data (with a reasonable limit of say 5 GB) for $20, the data traffic is 2000 Mb/$. Thus the intent of the bits, whether it is an SMS message, voice, or best-effort data, effects how it is billed. Thus AT&T's interest is to ensure that the iPhone can't circumvent intent-based billing.

Overall, there is a design philosophy which is creating a sealed box rather than an open box. The sealed box offers some better security properties (as AT&T theoretically does not have to worry as much about misbehaving iPhones), but the security properties are somewhat illusionary. Attackers will still be able to compromise the Safari implementation and gain control of iPhones. It will be difficult for attackers, but doable and highly attractive.

Additionally, the hole in the sealed box, the ability to run sanboxed Ajax-ish web applications, defeats AT&T's intent based pricing, the stated and implied security goals, and Apple's stated goal of a pristine user experience. An Ajax-ie webpage could easily interface with IM protocols, replacing high-value SMS traffic with lower value bulk-data. It is vulnerabilities in the web browser which attackers will exploit. And the interface will never be as good as a native interface running directly on the iPhone.

In the end, the iPhone is a porsche which can only turn left.

If you only ever want to do what Apple has decided you should do (namely email, web surfing, music, and a phone), it is a beautiful platform, and probably worth every penny.

If I could obtain development tools and install new applications, I would buy one in a hot second, even with the transition costs as a Verizon customer.

But with the current model of a sealed box, I will not buy one and will urge my friends and family not to buy one, at least until it costs no more than a basic phone. It may be beautiful, but it is crippled.

Personal Financial Security Protocols

2007-06-21T10:17:00.001-07:00

Note: The following is a work in progress. Comments are greatly appreciated.

There is an old saying, "The cobbler's children have no shoes", implying that experts in a field often neglect their own discipline in their daily lives. For me, as a security "expert", this is not the case. I have a rich and complex set of personal protocols for dealing with financial matters, including protecting my bank accounts, savings, and credit cards. I deliberately designed these protocols to balance security and convenience.

I began with a simple observation: I want to minimize my costs in a security breach. And costs to me can be reduced by either preventing security incidents or ensuring that some other party, not myself, is responsible for the lost. Thus my attitude towards my credit cards, my bank account, and brokerage account are all substantially different.

Credit Cards

I am generally rather cavalier about my credit card. I happily use online shopping, and will even email my credit card number when making a reservation at a small hotel. True, I'm not going to post the number on the Web, but I won't otherwise hesitate to use my credit card and don't take any extra care in safeguarding this information.

Why? Simply because it is not my money at stake!

Until I write the check to the credit card company, it is the credit card company's money. In case of fraud, I am able to dispute the fraudulent transaction before I have to write the check, leaving the credit card company on the hook for all but $50 (in theory) or $0 (in practice). I had this occur once, with a $5 fraudulent charge, and the process of disputing the charge was painless. Rather, it is the merchants who need to take care in accepting credit cards as the merchant ultimately carries the cost of fraud.

Bank Account and ATM Cards

My casual attitude towards my credit cards is sharply contrasted by my attitude towards my ATM card. My ATM card is ATM-only, without a Visa or MasterCard logo. With a "check" card, where the transaction goes through the credit card system, all an attacker needs are the numbers on the card. In contrast, the ATM network requires the PIN number as well as the card's information.

Additionally, I only use my ATM card at a bank branch's ATM (ideally my bank's branches). And even at these ATMs, I physically examine the slot where the ATM card enters to see if someone has attached a card skimmer (a device to read the card as it is inserted into the machine). I NEVER use my ATM card at grocery stores or other stores, as there have been several break-ins where attackers have managed to capture ATM cards as well as credit cards.

Why should I care? Although the fraud protections for ATM/check cards are as good as credit cards, until the dispute is resolved it is my money that is missing, not the banks. If someone fraudulently used my credit card, the worst case would be the card stops working (and I have two cards). If someone fraudulently accessed my bank account my rent check might bounce before I found out. Thus I need to minimize the chance of a breach.

I also do not use any automated or online bill pay or online banking, except for a couple which go to a credit card. My banking and bill payments are all done in person or through the mail. There are too many bots and key logger in this world for me to trust online banking and there is significant comfort in having a real paper-trail for any potentially disputed transaction.

Finally, when I do pay my bills by mail, I drop off the envelopes in a locked mailbox rather than leaving them for the postman to pick up. It is far too easy for someone to steal some checks and modify them if they are out in the open.

Brokerage Account

The one exception to the "No Online Banking" rule is my brokerage account, as the web site provides the only effective interface for managing the account. Fortunately I only need to access it once every few months, as I follow the general economic advice of "Buy index funds and/or CDs and just let them sit" as I know I'm incapable of reliably beating the market.

I use a bootable Linux "Live" CD (in my case, Knoppix, although I need to investigate alternatives as Konqueror doesn't render properly, forcing me to manually download Firefox). I reboot my computer using the live CD so I know that my system is free from viruses, bots, and keyloggers. I then access just my brokerage account, do my necessary changes, and restart my computer. Although significantly inconvenient, I view this as necessary.

Unlike bank accounts, the laws concerning fraudulent brokerage account access are not well-enough settled for my taste. Since I have no assurance that, in case of fraud, I would not lose money, I need to prevent fraud to as great a degree possible. Thus I must be able to trust the computer I'm using, and given the perilous state of end-host security (even Mac security), the only way I can trust the computer is by booting using trusted, read-only media and only connecting to the brokerage account.

Conclusions

Building these financial protocols took me considerable thought and effort. I had to consider what were the possible attacks on my financial data and what the consequences were. In the end, it was the consequences of possible attacks which dictates my policy: if it doesn't cost me much time and money, I don't care. But if its my money on the line, I'll be very careful.

Intro (Redux)

2007-06-20T09:53:00.001-07:00

A redux on my intro:

I figure that I'm enough of an egomaniac that I finally should start up a blog. After all, it is only academics with LARGE egos which should be blogging... This is not really very active yet, but I expect to use it in the future to post original items.

For background, my research area is computer security and computer architecture. I received my Ph. D. from UC Berkeley in the fall of 2003, and since then I've been a researcher at the International Computer Science Institute (ICSI).

This blog, however, will also include my thoughts on random topics of which I am completely unqualified as well as information on computer architecture and security topics.

I tried to start blogging, let it lie fallow, and am going to try to start again.

When should we start being really scared?

2005-08-07T11:11:00.000-07:00

As an non-economist, when should we start being scared?

1) We have a household savings rate of 0%.

2) Huge deficits in both current account and government, to the tune of several hundred billion a year.

3) A spookily-flat yield curve (my bank will loan me at <6% for 30 years, but will borrow from me at ~3.8% for just 9 months!), which says that some huge amount of long term money is amazingly optimistic.

4) A real-estate market in areas that is so horribly bubbled that tax-adjusted interest, property tax, and HOA/maintinence is vastly more (30%+) than rent.

5) and a government in total denial.

When should us normals start being really, REALLY scared? As Kent Brockman asks on the Simpsons: "Is now the time to panic?"

Simple Little Delay-Line Hack...

2005-07-28T12:35:00.000-07:00

People have proposed requiring the client system to do work as a way of limiting/mitigating DOS attacks, and others have countered that it isn't fair to small devices (eg, phones) as there may be 1-3 orders of magnitude difference in computing power between clients. Thus a followon proposal is many schemes which just force a client to WAIT when the server is under load. I've heard of some rather complicated schemes to do so.

There is a VERY easy way to do this, however:

The server, on startup, creates a random key.

When it gets a request from a client, and it wants the client to wait, it sends back a message saying "Wait x seconds + resend with this cookie". The cookie being E(K, time its allowed, IP). Now the client waits and resends the request with the cookie.

Voila, the clients wait for the specificed time, without the server having to store any per-client state or worry about any delay queue being filled. It requires only ONE encryption operation to create and one to verify, which on a modern CPU is only about a few hundred clock cycles.

So if your DOS-mitigation technique involves having new clients wait, this is all you need.

Passive Resistance to Stupid Security

2005-07-13T19:52:00.000-07:00

I have a great dislike for stupid security. Airline security in particular ticks me off. It's stupid. ID checks, pointless inspections of shoes, a complete ban on such deadly items as a pair of pliers...

At the same time, they don't screen the ground crews and maintenance staff, who can (and HAVE) smuggled a gun aboard the plane, shot the air crew, and caused a fatal crash killing everyone aboard. And if you print your ticket at home, you can easily eliminate the "flag me" text if you got unlucky and it says 'screen this person'. Or heck, change your name.

So I've been engaging in a minor campaign of passive resistance.

I have my driver's liscence or passport when I travel. But I don't show that anymore. Instead, I use my Lawrence Berkeley Lab ID card. It even says "Guest" on it in my job function. It IS official, issued by a US government lab. It even says so on the back, "Property of the US Government" etc etc etc. And it has a nice Department of Energy logo in the corner.

But the key is that it looks official. Airline desks, airport security, etc. I've used it at least a dozen times now, and I've only been challanged on it once. I would have fought the challenge (it IS a government issued ID), but it was a tight connection so I didn't want to play my normal games.

Now all I need to do is make up something that just LOOKS official. It just needs to have my picture, a good logo, and be printed on thick plastic. I'm thinking "Department of Bonehead Security", with an eagle bonking itself on the head to create the stars around it. Anyone challanges it, yeah, I'd whip out the driver's liscence. But until then, I'll have the nice plastic card.

Likewise, I hate taking off my shoes. I wear shoes with no metal. If the TIA guy says "I recommend you take off the shoes", I ask if I HAVE to. Sometimes the response is "if the metal detector goes off, you will get secondary screening". Fair enough: far too many shoes have metal shanks, and getting those people to take their shoes off removes a huge host of pointless false-positives. Sometimes its "We'll screen you, period". If anything, saying my low-cut hiking shoes are "too chunky". But the screeners have even objected to Tivas, so its obviously whatever the particular guy feels that day.

The other half of the time, no buzz, but secondary screening anyway. And you learn alot. Both times, they did NOT X-ray my shoes. Neither time did the TIA guy at the secondary screening know WHY I was screened. One time they wanded my wallet, the other time I simply held it out and it NEVER got wanded. The walkthrough detectors aren't sensitive enough to detect my cardkeys, but the wands are.

Both times, the TIA agent ordering the screening wasn't interested in security. If they were, they would tell the other agent why I was being screened instead of just sending me over to wait in another spot for the dude with the want. Rather, secondary screening is a punishment for questioning stupid rules. But hey, if I'm not in a hurry, its wasting their time, not mine.

All in all, airline security is a general exercise in silly security theater. But at least you can have fun with it.

Coming Soon...

2005-06-19T17:34:00.000-07:00

The disadvantage of doing a content-only blog, or attempting one, is that content CREATION is vastly harder than content referencing, especially when one has a day job.

Nevertheless, there are some upcoming rants/topics that I plan on pursuing in the near term. This is a preview.

"Stupid VM Tricks", or why you should hold off on infrastructure upgrades. How to leverage upcomming VM-friendly x86s and open-source software to build Windows networks with quick recovery, built in security primitives, ease of managemement, and easy patch rollout and rollback.

"Stupid VLAN Tricks", or why you should make sure that your switches are VLAN capable. The use of VLANs as sophisticated management and response tools for intrusion response and prevention.

"Home Users and Worm Defense". The one page of recommendations for home users to make their systems more secure.

"Consumer-Grade High-Tech Weapons". We have seen "consumer-grade" (cheap, plentiful) weapons (AK47, RPG) in the hands of our enemies. Might there be high-tech consumer-grade weapons? What might they look like?

"Attacking Document Collaboration". What changes should be made easy to use in Word/Word Perfect to prevent some pretty insidious attacks during contract creation/other collaboration with possibly hostile parties.

"Passive Resistance to Stupid Security". So much 'security' these days is ridiculous theater. ID checks and a fair amount of the airport security screening process is one of them. I'll describe some experiments in very simple, by the book passive resistance against these stupidities.

Should We Close Reagan National Airport?

2005-06-06T15:45:00.000-07:00

Matthew Dodd over at SFTT comments that a proposed policy allowing "private" planes to fly into Reagan National Airport represents another instance of Politics over Security.

I actually take an even more extreme view: I don't believe Reagan National should be open for ANY nongovernmental/nonmilitary flights.

The reopening of Reagan national to even commercial flights was a triumph of convenience over security: I guess senators didn't want to take a taxi from Dullis. The addition of "private" flights (read this as King Airs, Gulfstream IVs and Boeing Business Jets belonging to political contributors, not random Cessnas) is just an additional example.

My worry is not another hijacking to use a plane as a weapon, but an accidental (or ?faked?) deviation in flight which would cause the air defense systems around the White House and Pentagon to fire on a civilian airliner.

The landing aircraft pass so close to the White House that a quick-response air defense system must be in place to prevent a plane on final approach from being used against either the White House or the Pentagon. A flight deviation at the wrong time and some poor soldier is either going to have to shoot immediately or explain how he allowed a 737 to crash into the West Wing. Thus the air defense must be on a hair trigger during certain stages of a plane's approach. Mistakes can happen. And there could always be bug in the missile battery.

Yet imagine the disaster if the US military mistakenly shot down a civilian plane over Washington. Have the Iranians forgiven the US for the USS Vincennes shooting down Iran Air flight 655 back in 1988? Have the South Koreans ever really forgiven the Russians for Korean Air flight 007?

Lets say that an accident would be a One in a Million event for a given flight. With 800 commercial flights a day, that would be a 25% chance every year (1 - (1 - 1/1000000) ^(800 * 365)). Even if odds were 1 in a hundred million, thats still a .2% chance each year. Not wonderful odds, simply due to the sheer number of flights.

Combine both the non-negligible probability of such a disaster (there was at least one "near miss" with Kentucky Governor Ernie Fletcher's plane) with its impact, as well as the still existing possibility of a deliberate crash, and keeping Reagan National open becomes exhibit A in how security takes a back seat to the personal convenience of those running this country.

So What Will Happen To Real Estate?

2005-05-09T10:00:00.000-07:00

So what is my prediction? After all, I'm deliberately staying out of the real estate market, so why am I making this decision. The first question is how are people able to buy at all, with prices so high? With the example 2 bedroom condo requiring over $3000 a month in cash flow, and over $2200 a month in tax-neutral cashflow, how can anyone afford anything?

Of course, people aren't paying quite this much. The buyer of a house looks at the monthly cost more than the total value, and the monthly cost is greatly reduced through the use of adjustible rate mortages, especially ones with interest-only or negative amortization. And the statistics are showing that these have become increasingly popular. Yet even with those options its not a good deal to buy: at 3.5%, the tax neutral nonsavings is still no cheaper than renting.

And these mortgage carry a price: uncertainty. If interest rates on the ARM jump from 3.5% to 4.5%, thats an extra $200 in tax-neutral cost per month for my example. And long-term interest rates are unnaturally, incredibly depressed: Roubini and Setzer argue that interest rates are artificially lowered by 200 basis points, 2%, because of asian central bank currency intervention. So if George Bush gets his way and China floats its currency, say hello to a large interest rate jump.

Assuming Roubini and Setzer (and numerous others) are correct, and that the currency situation can't last forever, or something else will cause interest rates to rise, why get an ARM? If you are going to sell in 2-3 years an ARM is the way to go as the long term uncertanty isn't significant for a short-term loan. But this is the classic bubble assumption: prices keep going up (and it has to go up at least 6% to cover transaction costs when selling). Otherwise, to reduce payments for a longer time, its a huge gamble: If interest rates go up to 6%, its tied, and beynd that, its a catastrophe. Given long term interest rates so historically low, why take the risk?

Whats worse is that people are using these alternate mortgages in order to afford a house at all. If someone is squeeking by, leasing their car and with an ARM on their house, leveraged to the hilt, what happens if interest rates jump? If the economy goes south? There is now a huge number of people with no margin for error.

So what is my prediction: Well, interest rates are going to go up a little and the market will freeze: Buyers will stop buying as their monthly costs go up, but sellers won't lower their price. Taking my example, to have the same tax-neutral cost/month, a rise in interest rates from 6% to 8% requires a drop from $450,000 to $395,000 in the sale price of the condo. So a 2% rise in interest rates in my example requires a price drop of 12%, even with buyers willing to spend the same amount as they currently are.

Now a flat market for a few years would be a nice, best-possible hypothesis. But I worry that the soft scenario won't happen. Rather, what I believe will happen is that after a year or two of freeze, with interest rates going up (and the economy shrinks as the refinanced-driven spending disappears), is that the crisis will hit: some small number of people will be forced to sell, yet buyers will be unwilling to pay more per month than they currently are. With a drop of 10% or more, this might end the bubble-mentality.

The worst case scenario, with a 200 basis point (2%) or more rise in interest rates, would thus be a huge collapse in price, as the bubble assumption is proved horribly false. And the collapse may be severe: Given 8% interest rates, the selling price to have rent-equivelent tax-neutral nonsavings cost for my example (observed over 4 years) would be $250,000, a nearly 45% drop in prices!

Of course, I hope (even as a non-homeowner) a tokyo-level collapse will not happen, and would be very unlikely (I hope). Yet a 15% drop seems certain, and a 30% drop would not be out of question. If prices dropped 30%, even with higher interest rates, then I'll probably buy a house: it gives a huge hedge against inflation, the value will go up if interest rates go back down, and the price won't be so obscenely out of line when compared with renting.

(note, minor edits for clarity)

Why I'm Not Buying A House..

2005-05-06T15:43:00.000-07:00

As I've graduated, and now have a stable job and income, I'm supposed to think about buying a house, rather than continuing to pay rent. But with Bay Area home prices at dizzying levels, I have to ask whether it makes sense to buy or keep renting. Being logical, I decided to use everyone's favorite financial "what if" tool, a spreadsheet, to construct a model of the various costs.

But before I get to the spreadsheet, what are the costs? There's the obvious mortage payment, but there are also property taxes, insurance, maintenance, and homeowners association dues. There are also tax savings, both with and without AMT to consider. Rather than just one cost for ownership, I consider 3 costs: the raw cash-flow, the tax-neutral cost, and the tax-neutral nonsavings cost. The first is obvious: how big per month are the checks. The second value is the first reduced by the tax savings from deducting interest and property tax. The final, tax-neutral nonsavings, also excludes payments on principle.[1] It is the last two which are the key values: what you pay (some of which you get back as principle when you sell the house), and what you pay and don't expect to ever get back.

You would expect that the tax-neutral nonsavings cost to be less then rent. After all, you should be able to buy a dwelling, rent it out, and make at least some profit. And, assuming you have the cash and cash flow, if tax-neutral nonsavings is less than rent, you need to buy.

Unfortunatly, with the current prices, this is not the case. Lets use the real numbers. I live in the Richmond Marina Bay area (aka the Yuppie Prison Complexes). My rent is $1325/month for a 2 bedroom apartment [2]. Recently, another, almost identical complex with the same floorplans and the same construction started being converted to condominiums, at $450,000 to $480,000 for the 2-bedroom units. The only major difference is that the other complex has some fake lakes, which only serve to attract the migrating geese and the rotting fecal matter they produce.

There are a few other assumptions needed: tax rates, loan terms, insurance HOA costs and inflation. This spreadsheet assumes a very healthy income, with a 28% marginal federal tax rate, a 9.3% state tax rate, and a 1.2% property tax rate. The loan I assume as 6% fixed, 30 years, with 10% down: a good loan for the long haul. Insurance I assume as .2%, which is actually low: earthquake insurance in the area costs .3-.4% depending on coverage level. I set the HOA fees/maintenance to $250/month. Finally, I assume that both rent and HOA fees increase by 3% annually. Toss the numbers into the handy spreadsheet and out pop some terrifying numbers.

Buying instead of renting simply costs a fortune. Beyond the $3200 a month of cash flow (hello Top Ramen dinners), the tax-neutral cost is still an outrageous $2200 a month. The real shocker is the tax-neutral nonsavings cost: $1880 a month, or over $550 more than renting. That's a real loss of $550 a month, for the privilidge of owning a glorified 2 bedroom apartment (err, "condominium home"). I can buy a nice car for $550 a month. It takes 7 years of inflation (by which time I would save $34,000) before my monthly non-savings cost would be equal to rent. It would take 18 years! until the net cost is the same.

Note that I did not consider asset appreciation in this analysis. Mostly because I feel that this is a dangerous bubble, and over the next 5-10 years, prices are going to be, emm, interesting (more details in a subsequent post). But lets say I want to sell after 7 years. Prices will have to have gone up by another 13% to even break even compared to renting, considering the 6% cost of transaction when selling a house.

It simply does not make sense to buy in this housing market. With shacks in the Richmond 'hood going for $300k, I'm going to stick with renting. For others considering buying, in other markets, use the spreadsheet yourself.

[1] An important note: I do not consider asset appreciation in the model. When there is such a feeling of speculative bubble, I don't want to include speculative gains in the calculation. But I do assume, by considering the money paid to principle as savings, that the value won't drop below the down payment.

[2] Two other factors I'm also excluding: the complexes considered are in Richmond California (a truely atrocious school district) and are built on bay fill. In the event of a major earthquake, every dwelling in the area is going to be red tagged. As a renter, that just means I need to move my stuff. If I was an owner, even with insurance, its a catastrophe.

Export License Required to Log In...

2005-05-06T13:57:00.000-07:00

The Commerce Department, in the Federal Register, has proposed some significant changes to the Export Control Rules. The changes seem subtle and arcane (a change of 'and' to 'or', changing country of citizenship to country of birth OR citizenship (whichever is more restrictive), and a couple of "clarifications"). But the implications appear huge, especially the 'and' to 'or' change. Assuming I'm reading this correctly, it sounds like whoever allows a foreign citizen to use a supercomputer (or other export controlled device) has to get an export license and approval from the federal government. And just about every remotely decent cluster qualifies. Will universities be forced to deny access to Chinese graduate students? What if someone had the misfortune to be born in Iran? Or Cuba? It's not too late to submit comments (mail to scook@bis.doc.gov, with "RIN 0694-AD29" in the subject line), as the comment period extends until May 27th.

For export purposes, a "Supercomputer" is a system capable of 190,000 MTOPS (Million Theoretical Operations Per Second). The definition generally includes clusters of systems, not just individual computers. As the MTOPS is basically any instruction, at the maximum theoretical peak of every functional unit running as efficiently as possible, a normal computer actually scores very high. AMD conveniently publishes theses values, so a Dual Processor Opteron 248 is 15,000 MTOPS. Thus a cluster of only 13 $3000 Sun Fire v20z would be called a supercomputer and subject to US export controls. A computer lab where users can submit jobs to multiple systems simultaneously might also qualify.

Note, I initally saw this elsewhere today (I don't remember where), but I decided to actually look at the proposed rule. Yes, it is this scary.

Introduction

2005-05-01T09:04:00.000-07:00

I figure that I'm enough of an egomaniac that I finally should start up a blog. After all, it is only academics with LARGE egos which should be blogging... This is not really very active yet, but I expect to use it in the future to post original items.

For background, my research area is computer security and computer architecture. I received my Ph. D. from UC Berkeley in the fall of 2003, and since then I've been a researcher at the International Computer Science Institute (ICSI).

This blog, however, will also include my thoughts on random topics of which I am completely unqualified as well as information on computer architecture and security topics.