Why I think Exploit Markets are bad - a response to Roberto Preatoni of WabiSabiLabi

Dominic White

 

Why I think Exploit Markets are bad - a response to Roberto Preatoni of WabiSabiLabi

Last week, Roberto Preatoni, founder of WabiSabiLabi, the exploit eBay, gave a talk at the ITWeb Security Conference about his creation. I really wanted to ask a question, but there was no time. At the end of his talk, when asked who agrees that WabiSabiLabi is a good idea (i.e. creating a market place for vulnerabilities and exploits to be freely sold and traded, like eBay) like Roberto, I was surprised to see so many hands go up as the general info sec community has reacted quite harshly to the idea. A possible explanation based on my experience is that many ITWeb attendees are not 'hardcore' security people, and haven't been following the disclosure argument over the last decade. Then, given only Roberto's talk, chose to agree with him due to a lack of any exposure to rebuttal. When asked who disagrees, I was the only one who put their hand up.

So, here's why.

First up, let me say, I don't know Roberto personally, and have no personal vendetta. I also do not work for a security vendor, so am not coloured by a shoot the messenger mentality. Finally, I agree that to do what Roberto and his team did took serious courage and they have weathered some big (and unfair) fights, which probably would've had me packing for the hills. However, from the comfort of my blogging seat, I don't think an exploit market is a "good thing". My rebuttal *is* coloured by the ethics debate. It could be a hangover from the last decade of the debate being coloured by this, but, as a 'white-hat' I am specifically defining myself as a security person with ethics. If you don't care about anything but getting the most money to vulnerability researchers, then you won't buy my argument.

Roberto's main premise seems to be that vulnerability researchers (I don't like the term 'security researcher' for this purpose, it is too vague and generalised), are forced to hand over the results of their time and effort for free, to a vendor who often does not respect them for their work, all in the name of ethics. By providing a market-place, a place where sellers and buyers can gather, vulnerability researchers can have their work monetized. The implication being, that vulnerability research can be a job, and the security world benefits by getting holes closed.

The All Knowing Market

I've never been a fan of 'free markets'. The main reason is that the invisible hand isn't always an ethical one. Take for example the world's terrible food distribution, where food is literally thrown away while others starve all because of how the market has structured the whole scheme to get the buyers with the most money to the sellers. Roberto explicitly mentioned the pharmaceutical industry as a comparison, which is a great example of where the market hasn't provided for the needs of people. We've had people researching esoteric illnesses instead of the African killer, tuberculosis, because of where the money is coming from. We've had over priced AIDS drugs, forcing India, Brazil and SA to threaten calling a state of emergency to allow us to produce cheap generics in terms of the TRIPS agreement. The end result is that the best course of action, namely curing the most number of most sick people, hasn't happened, but is considered better than no market where doctors do it for free. There are 'better' solutions however, such as the medical innovation prize fund, a model which could well work here, but more on that later. Let's make it more specific though, here's why I think a free market in this instance is bad too.

Implications

Ok, so given that by creating a market, things may not be driven by ethical lines, let's examine what sort of behaviours we could see.

Vendors held ransom

Now that the bugs are being peddled on an open market, vendors must compete with a variety of buyers, from security firms to bad guys. They will always have to assume that their buying the bug is better than someone else buying it, and in the case of big bugs (Vista 0day for example) will have to pay a fair amount of money. This has two problems. The first is that vendors have to start spending money that could otherwise be used to find, fix, or better, prevent, their own bugs. The second is that you have a skewed market where it is always in the best interest of the vendor to acquire the bug, and they are pitted against the whole market, not exactly willing buyer.

Criminal Syndicates

Given that the vendor now has to outbid the entire market, it is likely that prices will be driven fairly high. Now, we can safely assume bad guys may want to buy bugs. However, the individual bad guy wont have the funds. Thus, bad guys will need to club together and pool their money to outbid the vendor (or other security firms), and now you have a syndicate. I'm not saying that this is the only way syndicates can form, just that it encourages new ones to form, and makes existing syndicates likely competition. WabiSabiLabi does implement a vetting system, but a quick look at existing handgun registration systems will show that this isn't an effective way to stop bad guys from buying legitemate (as opposed to the black market guns we know they get), or at the very least, from using them to kill someone, it is just a semi-ineffective post-event auditing technique.

Black Market

On the syndicate point, people often suggest free or free-er markets in reaction to the existence of a black market. This could provide some interesting stats; we could get an idea of what bugs are being traded, how many, their price etc. A potentially very valuable tool for threat research. However, Roberto himself concedes that this will not get rid of the black market. First, because many syndicates are actually hiring vulnerability researchers of their own (some straight out of varsity) and second, because the existing black market provides more anonymity, or at least more trusted anonymity. In essence, they have their own vulnerabilities, and can make more money exploiting things with them, than selling them.

Weaponisation

Another implication of this market is that it will encourage people to weaponise vulnerabilities into exploits. Right now, (for example) a demonstration of a repeatable crash is usually enough to show the vulnerability exists, and things are weaponised usually to prove a point. However, in a true exploit market place, several types of guns (exploits) can be sold to exploit the same vulnerability in the human head. Thus, when a Microsoft patch is released, it may be worth my while to reverse engineer it and create a working exploit to sell to the highest bidder. This could be stopped by only allowing original vulnerabilities to be sold, but seems like an arbitrary restriction on the otherwise free market.

Arbitrary Restrictions

Speaking of arbitrary restrictions. Roberto brought up a few in his talk. He mentioned that they refused to sell exploits in online poker sites, or an XSS in MySpace etc. The general trend was that they didn't allow web app vulnerabilities to be sold. This is a pretty arbitrary restriction, as I don't see much difference between allowing an authentication bypass for windows to be sold vs allowing an authentication bypass for MySpace to be sold. If you look at other 'market regulators' such as the SEC or our own JSE, they usually implement controls to ensure the market functions correctly (e.g. prevent people from misrepresenting market related information e.g. financial statements or abusing market mechanisms e.g. insider trading). They do not stop certain stocks from being traded based on the supposed actions of the investors or the company. Now, if WabiSabiLabi has already started down this road of arbitrary restrictions, most likely due to some ethical drive, then they must acknowledge that they want an ethical outcome. Given that a free market doesn't lead to an ethical outcome, a free marketplace may not be the best way of doing this. In short, they either have to remove the restrictions and put up with the potential badness, or keep them and be labeled hypocrites or at least inconsistent (logical doomsday words ;) ) or shut down the market.

A better way

In his talk, Roberto did a sort of reductio on responsible disclosure. However, the model of responsible disclosure he presented was the sort of model 3Com's Zero Day Initiative (ZDI) takes. I was fairly confused about this, as I don't see much difference between the ZDI model and that of WabiSabiLabi's in terms of disclosure, in fact Roberto acknowledged them as competitors.

Now, responsible disclosure isn't really a well pinned down term, but I don't believe it is an inherently flawed model in all possible cases. So let me present a model as I believe many understand it, which will work to the most ethical outcome, and cover Roberto's need to monetize the whole shebang. It will be brief, as there is already stacks of info out there about this.

If you find a bug, report it privately to the vendor. The vendor should get back to you within a reasonable time frame, produce a good fix within a reasonable timeframe and credit you in the patch release. If the vendor is antagonistic or takes too long to release a patch, then full disclosure applies where all details are released publicly to drive the vendor to action. This model worked very well with Microsoft, where full disclosure drove the otherwise antagonistic or ignorant vendor into making some serious changes and turning them into (dare I say it) the good security vendor we see today.

This model is not without money. The disclosure model gets you credit, this has long been a way of building up eminence in the market. Over the long term this credit gets your day job work. Take David Litchfield as an example (who incidentally spoke before Roberto). He flogged Oracle silly over their terrible security, and now is probably one of the highest paid Oracle security consultants in the world, and even has Oracle as one of his clients. This is a man who monetised his vulnerability research. Granted, he couldn't just sit in a chair making Gobbles-like vulnerability findings and hope to make money, but it certainly gave him the opportunity.

Vulnerability Research Prize Fund

To be honest, I haven't put much thought into how the medical innovation prize fund described here, here and here could be retrofitted to this model. Although, my gut feel is that it can given the similarities in the vulnerability research field to the pharmaceutical industry, i.e. we want to pay for and encourage the best research, but we want the knowledge to be shared as widely and appropriately as possible instead of creating monopolies (e.g. ZDI, Syndiactes). It would be great for some of use to put some thought into this and see what we can come up with as it would potentially provide an 'even better' way that any of the exisiting solutions.

Conclusion

So, at the end of it all I believe that the vulnerability market place won't lead to the best outcomes, already has some inconsistencies and may slippery slope into more, and that a better model already exists. Finally, I believe an even better model could exist structured along the lines of the medical innovation prize fund and that more attention should be paid to this as a solution.

Thursday, May 15. 2008
Posted by Dominic White in Security
Comments (7) | Trackbacks (0)

 

Trackbacks
Trackback specific URI for this entry

PingBack
Weblog: singe.za.net
Tracked: Feb 05, 09:23

Comments
Display comments as (Linear | Threaded)

#1 Julius Francis (Reply)
Posted on Friday, May 16. 2008
Thank you Dominic for your appraisal of the Preatoni talk. I agree with many of your assertions and I am also not associated with any security vendor. Security Practitioners should to be discerning and introspective with regard to their actions. A good rule of thumb is to ask oneself the question: "Will the actions that I am about to take make things more or less secure?" If the answer is the latter, then it needs to be treated with circumspection. Unfortunately, in Preatoni's mind, the criminal cracker euphamistically becomes a "security researcher", much the same as a whore becoming a "Sex Care Worker". Preatoni is simply a cheeky thug who advocates his actions by transferring the focus to the vendors. Not that they are angels, but this kind of advocacy strategy is well known amongst the worst criminals. Even on his dying day, AL Capone believed that he was a "good guy" who provided amply for his employees and made many men wealthy. There is good reason for the US Army to be in the "top ten" visitors to his "marketplace". They have a cell in Guantánamo Bay with his name on it. PS: You were not the only one with your hand up.
 
#1.1 Dominic White wrote (Link) (Reply)
Posted on Friday, May 16. 2008
Hey Julius, Thanks for the support. I've heard a few people say they had their hands up too, so I apologise for my statement. However, I'm not sure if I agree with the idea that all vulnerability research is a criminal activity. Take David Litchfield as an example. His good work is what pushed Oracle security forward by a couple of decades, and I'd argue he was *too* careful about it. For example, he waited 3 years before he disclosed one bug Oracle hadn't fixed. Also, please be careful of name calling, it tends to turn these sorts of discussions into fights. On your point about the US army, and Roberto's about IDS signatures. It is interesting to note that Microsoft gives the Air Force copies of their patches before anyone else in the world. Given that a patch can be reverse engineered to an exploit (more reliably than an IDS sig can) - Halvar Flake did it in 20min with his BinDiff plugin - this potentially gives the US .mil monthly 0days and a large cyber-offensive capability. P.S. Psalm 37 is a nice one, thanks.
 
#1.1.1 Julius Francis (Reply)
Posted on Monday, May 19. 2008
--->"...I'm not sure if I agree with the idea that all vulnerability research is a criminal activity." Hmm, I don't recall stating that. (Reads response again.) No, I did not state that all vulnerability research is a criminal activity. My only reference to the word "criminal" was when I described the state of mind that euphamizes criminal activity by obfuscating facts. It's quite a common state of mind, actually -- hence my reference to Capone. ---> "Also, please be careful of name calling, it tends to turn these sorts of discussions into fights." Touché. I have a tendency to be outspoken on matters that I feel strongly about, and tend to call a spade a spade. Then again, this is your blog and I'll respect your rules. ---> "... this potentially gives the US .mil monthly 0days and a large cyber-offensive capability." Agreed on the facts, however I am undecided on whether this is a good thing or not.
 
#2 Roberto Preatoni wrote (Link) (Reply)
Posted on Friday, May 16. 2008
Hello, Dominic, the power of Google brought me here. I am Roberto Preatoni, and yes, I was also surprised to see that you were the only one raising your hand. To be honest, I was curios to see "who was that guy" and to hear his motivations eventually opening up a discussion thread. We can do it here, that's the beauty of the Internet. Point by point, as you wrote: The All Knowing Market We should distinguish between the world as we would all wish to see and the real world. In an utopian world, famine and diseases are solved by humanitarian-driven actions. In the real world, everything is tighten to money. I know it's not nice, but that's the way it is. And if it is in that way, it's just because the world it's driven by us humans. Do doubt my dog, being it in power, would choose to drive the world under the flag of justice and ethics. Vendors held ransom I agree, the market competition eventually will hold vendors ransom. But again, in a free market-driven approach, when the vulnerability's price paid by the vendors will be pushed so high, it might be the proper time for them to decide to invest the appropriate amount of money and efforts in security research and development. And that's the natural conclusion of all the discussion: the vendors should carry on ALL the costs related to their product security. Period. Black Market No matter what we will do, what actions we will take, there will always be a black market, ready to pay high prices for vulnerabilities. We must live with that, it's just ethical dreams against pragmatism. Weaponization. As I said in the conference, the easiest way to weaponize a vulnerability is to buy a IDS/IPS and to some reverse engineering work. Let's put it in this way: Responsible vendors = secure software = no patches = no weaponization Up to that point, no practical discussion can hold security researchers responsible for the eventual weaponization of their findings. It's like if you tell me that the mining industry should account responsibility because somebody used a knife made with its metal, to stab someone's back. Arbitrary restrictions: You made a mistake. We do allow web app vulnerabilities in our marketplace (in fact, we had several listed in the past). What we do not allow, is a vulnerability related to a specific online service (website) as that info could not be useful to the general public or legitimate penetration testers, but would harm directly the website and it would be interesting only for those who have criminal intents. A better way. Another mistake. We, in fact, do not apply the responsible disclosure policy, as ZDI. We provide just a marketplace. It's up to the security researchers whether to disclose it or not to the vendor, we just comply to his directives. Vulnerability research prize fund. Here I might agree as an alternative model to compensate researcher's efforts, but only if the prize would be paid by the software vendors and if the prize would be hefty, not just some peanuts as it used to be in similar initiatives. An example? The prize given by the Mozilla foundation for a new vulnerability under its Bug Bounty contest: 500$ and a laptop bag. Excuse me, sir. Best regards, Roberto Preatoni PS to Julius Francis You statement: "Unfortunately, in Preatoni's mind, the criminal cracker euphamistically becomes a "security researcher", much the same as a whore becoming a "Sex Care Worker"." was delightful. The whole security researchers category is now pleased to know that they are just "criminal crackers". So kind of you, cheers.
 
#2.1 Dominic White wrote (Link) (Reply)
Posted on Friday, May 16. 2008
Ciao Robert, I'm so glad you found your way here. The internet is indeed a wonderful place. I tried to find you at the conference to have this chat face to face, but this will have to do. Jumping straight into it, let me respond: The all knowing market ---------------------- I realise money is king, but being able to apply hindsight to ensure the money is spent in the best way on the best outcomes is what I am advocating. Not creating a market for a market's sake (although I don't believe that's what you're doing.) Vendors held ransom ------------------- But that's the problem. It isn't really a free market if the buyer isn't willing, and are forced into buying it. Additionally, the high price will need to be paid in addition to any security work they do, making smaller vendors less likely to be able to afford it. Finally, there is already a way to force vendors to invest in security work, full disclosure. It has worked well in the past (although not ideally). Weaponisation ------------- I don't believe IDS/IPS signatures are that easy to weaponise. Very few instances will allow a few bits of packet info to fully reveal an exploit. At best it could point out where to look for a bug, but it is unlikely we will see automated exploits generated from IPS signatures. Arbitrary restrictions ---------------------- Thanks for elaborating on your criteria. However, if you are worried about affecting one merchant, why are you not worried about potentially affect many? As the only difference between a bug that exploits a single vendor vs. a bug that exploits a platform in this sense is the affected population? Also, this still doesn't address my main point of this section, that the restriction is arbitrary when put next to the principles of a 'free market' and leads to the sorts of inconsistencies I've just pointed out. A better way ------------ I agree that there are large differences between ZDI and yourselves. But in terms of disclosure there isn't much, because anyone who buys the bug can keep it secret. A vendor who buys it can refuse to ever disclose it potentially leaving us as users vulnerable like the used to before the good ol days of full disclosure. The fund -------- I'm still rolling the idea of the fund around in my mind. There are a variety of ways it could be funded, and I agree the price would need to be large enough to incentivise real research. I would love you to look through it and give your thoughts, as you're probably the best person to. That's about it. Thanks for the reply, and we hope to see you in SA again soon. We'll make you some proper putanesca ;)
 
#2.1.1 Roberto Preatoni wrote (Link) (Reply)
Posted on Friday, May 16. 2008
Vendors held ransom ------------------- --->"But that's the problem. It isn't really a free market if the buyer isn't willing, and are forced into buying it. Additionally, the high price will need to be paid in addition to any security work they do, making smaller vendors less likely to be able to afford it." Here you fall in some contradictions. The same full disclosure was enforced because it was clear that NO VENDOR was willingly put any effort in patching the holes. Let's not forget this problem, which is the very foundation of the whole insecurity issue. --->"Finally, there is already a way to force vendors to invest in security work, full disclosure. It has worked well in the past (although not ideally)." Which means, security researchers should work for free to force the vendors to produce patches. See, the beauty of the WSL project is that it doesn't want to replace the current models, it just want to be an alternative. If a security researcher wants to comply to the full disclosure policy instead of referring to our marketplace, we praise his choice and we support it. I personally did it in the past and I know some researchers that, depending from case to case, might decide to go for the full disclosure process or to go through our market. They are free, they have a free will and we respect it. Weaponisation ------------- --->"I don't believe IDS/IPS signatures are that easy to weaponise. Very few instances will allow a few bits of packet info to fully reveal an exploit. At best it could point out where to look for a bug, but it is unlikely we will see automated exploits generated from IPS signatures." Uhmm... the guys here at WSL's laboratory smiled a bit ;) Arbitrary restrictions ---------------------- --->"Thanks for elaborating on your criteria. However, if you are worried about affecting one merchant, why are you not worried about potentially affect many? As the only difference between a bug that exploits a single vendor vs. a bug that exploits a platform in this sense is the affected population? Also, this still doesn't address my main point of this section, that the restriction is arbitrary when put next to the principles of a 'free market' and leads to the sorts of inconsistencies I've just pointed out." Once again, you are referring to merchants, we are referring to software producers. Two different categories. A better way ------------ --->"I agree that there are large differences between ZDI and yourselves. But in terms of disclosure there isn't much, because anyone who buys the bug can keep it secret. A vendor who buys it can refuse to ever disclose it potentially leaving us as users vulnerable like the used to before the good ol days of full disclosure." An example: a security researcher places his findings on WSL's marketplace. WSL ask him wheter he wants the vendor to be notified or not. The researchers chooses to notify the vendor. Still, his exploit is on the marketplace and me, as a security penetration tester, I might want to buy it to offer services to my clients, because I know that the vendor will take some time to produce a patch therefore i'll have a time window to market my services, even if the hole was disclosed to the vendor. Make sense? The fund -------- --->"I'm still rolling the idea of the fund around in my mind. There are a variety of ways it could be funded, and I agree the price would need to be large enough to incentivise real research. I would love you to look through it and give your thoughts, as you're probably the best person to." I certainly will. How about this: what about a law, who forces software vendors, to set a certain percentage of their incomes generated by software sales to be given to such "security funds"? After all, whenever you buy an insurance policy for your car, the insurance company is forced by law to set part of their incomes to be used to purchase stocks or real estate, as a guarantee fund for the policy subscribers. --->"That's about it. Thanks for the reply, and we hope to see you in SA again soon. We'll make you some proper putanesca ;) Excuse me, I'll cook with pleasure for ya'll.
 
#2.1.1.1 Dominic White wrote (Link) (Reply)
Posted on Friday, May 16. 2008
--->Here you fall in some contradictions. The same full disclosure was enforced because it was clear that NO VENDOR was willingly put any effort in patching the holes. Let's not forget this problem, which is the very foundation of the whole insecurity issue. Sure, but given that we have full disclosure, why introduce another alternative that has potentially worse outcomes? --->Which means, security researchers should work for free to force the vendors to produce patches. See, the beauty of the WSL project is that it doesn't want to replace the current models, it just want to be an alternative. If a security researcher wants to comply to the full disclosure policy instead of referring to our marketplace, we praise his choice and we support it. I personally did it in the past and I know some researchers that, depending from case to case, might decide to go for the full disclosure process or to go through our market. They are free, they have a free will and we respect it. I agree with you here, it is an alternative, and no-one is forced to use it. However, I am discussing the implications of if it were to 'take off' as WSL seems to be. As for researchers working for free, like I said in the original blog entry, I think there are ways that valuable research can be monetised without needing to trade the potentially dangerous good to the highest bidder. --->Uhmm... the guys here at WSL's laboratory smiled a bit ;) I am not saying it will never work. Take for example, SQL slammer, a generous examples where the IDS signature gives away the bug. However, to turn that into an exploit you still need to merge the payload and bug into something clever. But as you move onto more complex things, your ability to even discern the exploit can become difficult. Taken to an extreme case, you don't get IDS signatures for local privilege escalation, let alone an ability to reverse engineer it. Although, I'm prepared to admit that you may have evidence to prove me wrong, can we see it ;) --->Once again, you are referring to merchants, we are referring to software producers. Two different categories. Merchants buy software from software produces. By implication a bug in that software affects all merchants. A bug in a merchants software affects only one merchant. software producer sploit population > merchant sploit population To use an analogy. If I sell a way to break into your house, someone can use it to break into your house. Whereas if I sell a way to break into door type x1, someone can use it to break into all houses with door type x1. It is inconsistent to sell one and not the other. Although, I admit you are more likely to get sued for selling merchant sploits. --->An example: a security researcher places his findings on WSL's marketplace. WSL ask him wheter he wants the vendor to be notified or not. The researchers chooses to notify the vendor. Still, his exploit is on the marketplace and me, as a security penetration tester, I might want to buy it to offer services to my clients, because I know that the vendor will take some time to produce a patch therefore i'll have a time window to market my services, even if the hole was disclosed to the vendor. Make sense? Thanks, that does make sense. I didn't realise you allowed the researcher to choose for the bug to be disclosed to the vendor and still be sold. Doesn't that reduce the value significantly though? My gut feel is that ticking the 'disclose to vendor' box would so badly reduce the price of the vulnerability, that people are incentivised not to? --->"I'm still rolling the idea of the fund around in my mind. There are a variety of ways it could be funded, and I agree the price would need to be large enough to incentivise real research. I would love you to look through it and give your thoughts, as you're probably the best person to." --->I certainly will. How about this: what about a law, who forces software vendors, to set a certain percentage of their incomes generated by software sales to be given to such "security funds"? After all, whenever you buy an insurance policy for your car, the insurance company is forced by law to set part of their incomes to be used to purchase stocks or real estate, as a guarantee fund for the policy subscribers. Could work. I don't see legislators going for it though, but I do believe that 'percentage of income' is a better way to ensure security than balanced scorecards. --->Excuse me, I'll cook with pleasure for ya'll. I look forward to it.
 

Add Comment