Archive for July, 2009

Rudd’s Essay in the SMH

Rudd’s Essay in the SMH

I found it pretty readable.  I’d invite everyone to read it.  Steve Keen approves of it, although Ross Gittens doesn’t (SMH Monday 27/7) (I trust Steve Keen more).

Steve Keen emphasises that ‘deleveraging’ (ie paying down debt) reduces GDP by the amount that is being paid off – eg paying $1 extra off your mortgage means not paying $1 to your local grocer or whatever.  The economy’s growth therefore starts off negative as it has to overcome the money being put to non-productive use paying off debt.   So if 3% is taken out of the economy because people are paying debt, then the economy must “grow” at 3% to just stay static.

In Keen’s post he identifies the last two great depressions (1890 and 1930).  In the first he notes that deleveraging occurred at an annual rate of about 4% – which he takes as a ‘natural’ rate of deleveraging.  In the second debt fell faster as a result of gearing up for and fighting WWII.  In that case debt fell by an annualised rate of 8%.     Australians have so much debt it will mean that either we reduce GDP by a small amount for a very long time (slowly deleveraging), or we reduce GDP by a large amount for ‘short’ time.  The very long time (based on 4% “natural” deleveraging) equates to 30 years before the debt to GDP ratio will be low enough for deleveraging to stop.    Here’s the kicker though – the ‘short’ time, based on 8% deleveraging is 15 years.

Keen believes that Govt stimulus will not overcome the massive debt that needs to be paid off.  He forecasts debt forgiveness as the only option which will work in any reasonable amount of time.

Microsoft contributes code to the Linux Kernel

Chris Smart reports that Microsoft has contributed code to the Linux Kernel… under GPLv2.  I guess this means they will now have standing to sue people for GPL violations in respect of that part of the Linux code they contributed?

Other coverage: Greg, Steve, lkml

Real World Test – Wdiff Best

Wdiff Compare – Best

Brendan Scott, July MMIX

[update Dec 2010: click here for my attempt at an online comparison service.]

(See also: docdiff post, wdiff post, OOo post)

I have earlier commented on the efficacy (or lack of it) of OOo’s compare function.  In my earlier posts I reviewed some of the comparison solutions which are available and are also free software.  One of the most promising of those was wdiff.  I did an initial review on some sample text but have not, to date, made any post on any real world use of these solutions.  This is that post.

I had four text files of comparatively large (about 10-12,000 words – or about 30 pages – each) agreements which I needed to compare.  This comparison was ‘live’ in the sense that the comparison was needed in the course of my review of these documents for a client.  Unfortunately, this also means that I’m not at liberty to post comparison examples since the documents are confidential (the comparisons in the earlier posts were run on a sample clause that I created).

Running Word compare on one pair of agreements  was a disaster.  Word seems to get to a certain point in the documents and then fall over dramatically (something I have witnessed in other situations).  For example, minor changes in a paragraph would make the whole of the balance of the document (or a large portion of it) marked as a change.  In order to get a useable comparison with Word I needed to run a comparison, then extract the section on which it failed, then run compare on those sections and so on iteratively. In fact, in the end I needed to manually extract most of the clauses either individually or with a small collection of clauses and (manually) run a comparison because Word repeatedly failed even on the shortened extracts.  This ran to about 20 or 30 separate extractions/comparisons using Word and was, frankly, pretty painful.

On a different pair of contracts I used wdiff.  Wdiff had similar problems to Word in that it got to a certain point at which it began marking the whole balance of the document as a change.  Wdiff’s raw performance was superior to Word’s in this respect, although the initial output, without any manual intervention was still not useable practically.  I could have used a similar strategy to that for Word of systematically extracting failed sections and re-comparing them.  Wdiff, however, had a significant advantage over Word in that it was able to be automated pretty easily (I used Python).  I wrote some scripts to compare specific clauses. That is, compare clause x of agreement 1 against clause y of agreement 2, output the results in html markup (see note 1) and join them together.   The results were fantastic.  Following this strategy, with clause by clause comparison wdiff didn’t get confused and produced a very good markup of the whole of each of the documents.  The downside was that it was not fully automated, but the results justified the comparatively small amount of manual input I needed to make.

Indeed, the automation allowed me to do some interesting things like compare different clauses within the same document (eg compare clause 1.1 against 1.2, 1.3, 1.4 and 1.5) to see how they were different.  Openoffice doesn’t properly render the html which is output though (it doesn’t show strikethrough) so Firefox needs to be used for display.

On that basis I’m revising my earlier rankings as follows:

wdiff – 8.5 (revised up because it is easy to script and this results in unexpectedly useful benefits)

Word – 7 (revised down because the markup goes feral repeatedly on the long documents I compared)

I did not revisit Docdiff, although I suspect it could also be automated in a similar fashion.

Release of IFOSSLR

Release of International Free and Open Source Software Law Review

Today marks the (soft) launch of a new journal dedicated to Free and Open Source Software legal issues (a “hard” launch with printed copies is happening in London on Wednesday).   The  International Free and Open Source Software Law Review is an initiative of the freedom taskforce run  by FSF Europe.   I am on the editorial panel and am also happy to take credit for initially floating the idea of having a journal.  I am also happy to praise the enormous effort that has gone into putting the law review together by the other members of the editorial committee and to the sponsors for supporting it.

There is, I think, a great need for a journal like this dedicated to free and open source software related legal issues.  At the moment, these issues remain hidden among other more generalist law journals – and are likely to be country specific.  Moroever, such articles are unlikely to be appropriately licensed.  Authors who are supportive of free and open source software are put in the ironical position of having to forego their (and their readers’) freedom if they want to publish articles on the topic.  The IFOSSLR rectifies these problems.  It will allow relevant articles to be grouped in one convenient place.  It will allow exposition of the cross-jurisdictional issues which are particularly important for communities which are by  nature multi-national.  Finally, its publication policy requires that all of the articles be openly licensed. In  general this means CC BY or BY-SA – so go download yourself a copy and share.  I expect to see it at the top on the torrents soon…

Pope on IP: Repent! Repent!!!

Patently O reports that the Vatican is coming out against monopoly rights over intellectual endeavour.  Quoting from an encyclical:

“On the part of rich countries there is excessive zeal for protecting knowledge through an unduly rigid assertion of the right to intellectual property, especially in the field of health care.”

!(Openness) && Standards Australia

!(Openness) && Standards Australia

Tom Worthington reports on an arbitration between Standards Australia and SAI Global over an agreement entered into in 2003.  The results of that arbitration are apparently that “Standards Australia must not permit or knowingly allow SDOs [Standards Development Organisations] to develop Australian Standards without securing for SAI Global exclusive rights to publish, distribute, market and sell those Australian Standards.”   And, of course, if you want a standard to become an Australian Standard you need Standards Australia’s accreditation.   I’m sure you can see where this is heading…

Subject to SAI Global granting an open licence over a standard, it seems that the set of standards accredited as an Australian Standard and the set of open standards will be empty.

Ooo Rescuing Word

Ooo Rescuing Word

Every once in a while my version of Word – for no apparent reason – chokes on a document and crashes whenever it is opened.  This happens every so often (- maybe once a month?).  The solution I have found works best is to open the document in Ooo (which so far hasn’t failed) and then save it again as a .doc.  Word is then happy to open it.  Tch!

It is ironic that I need to use an open tool to recover data from a closed format when the closed program specifically designed for that task falls over – especially given that the closed source vendor refuses to provide visibility of the data format.

WP on Annoying things

WP on annoying things

Ah, so true.

“With The Utmost Respect”, the Model is Broken

“With The Utmost Respect”, the Model is Broken

At this very moment, the only reason the West has news of any kind of the turbulence within Iran is due to the individual initiative of members of that community – powered by the Internet.   Rather than celebrate the wonderous power of free riding, driven by the Invisible Hand to overcome the concerted effort of government censorship Gary Becker and Richard Posner have chosen to post an article on their blog wringing their hands over the likely demise of the newspaper industry.   Their argument seems to be that, precisely because newspapers are becoming increasingly irrelevant and cost-ineffective,  the legislature should step in and disrupt the rest of the economy in order to perserve them.  The solution they pose is that consumers should pay more in order to preserve an inefficient industry past its use by date.  This is no solution at all.

In essence, their proposal is to take a leaf out of feudalism, randomly taking rights from citizens for the benefit of Newspaper Barons.   The right they propose to take is, quite literally, the right to give directions to others.  Under their dystopia linking would be illegal – that would be like making it illegal in Real Life to tell someone where the nearest school, or hospital is.   Such a proposal in the real world is so exceedingly bizarre no one would have the courage to float it in public, let alone posit it as a serious option.   That such a proposal can today be put forward at all indicates not only how completely disconnected from reality has copyright ideology become, but also how far that ideology, with its unhealthy obsession with demonising legal, justifiable, laudable free riding has permeated “official” opinion.    It is also testament to the far reaching power of copyright feudalism.

If newspapers serve no function they should be left to rot, or be shot.  The people they employ should be assisted to transition into a new world where they get paid like everyone else.   Some will thrive and some will falter and these are unlikely to be the same as thrived and faltered under the old model.    The world survived well enough before newspapers and I cannot see any reason why the world will not fare equally well in a future without them.  As is the case for news from Iran at the moment, there is no reason to think the absence of newspapers will result in the absence of news, or of opinion.  I did not go to a newspaper to read Becker and Posner’s article.   I read it where they published it – on their blog.    I went to them because I trust them to have an opinion as experts (notwithstanding that I find their approach affronting).   They’re both clever men, whose time is valuable.   Somehow they don’t seem to notice that they have managed to publish a piece of some non-trivial effort, despite not being supported by advertising (if it is there it is pretty subtle).  Depressingly, they serve as an obvious counter-example to their own argument.

Moreover, I can’t see my quality of life changing noticeably if I were never to read another newspaper (Nassim Nicolas Taleb has a similar view).   I do not, for example, have any newspaper in my list of rss feeds.  This is not to say I won’t have news or opinion, but, rather, that I will get it from a better source than a newspaper.   The “news” of today is such that it can be summarised in a ticker along the bottom of CNN.   To lose it is no loss.  The news of tomorrow will be more tangential and more relevant, closer and more distanced, more nuanced and more blunt.  It will be all those things because individuals are all producing news and perspectives on everything.   It is astounding to realise that there still exist people who think content creation stops in the absence of some Media Baron guiding it.

Becker and Posner’s proposal is not so much a proposal in favour of newspapers, as an attack on the individual rights of journalists.  Rather than saying that journalists ought to be able to compete with each other in an open market, they are in effect crueling the free market and, as a consequence, requiring that journalists be indentured to newspapers.   The natural consequence of their proposal is that a small number of places will become valuable as aggregation sources.  Over time it will cost journalists dearly in order to have access to a market through those aggregation sources.   The Becker-Posner proposal would be better placed in those societies which believe that individual initiative should count for nothing, and that the only actions individuals should embark on are those permitted by their betters.

No doubt they would charge for a link to the place where you get a linking licence.


[PS:  5 Feb 2011: the answer to their criticism about how to fund investigative journalism now seems somewhat ridiculous in light of Wikileaks – which has demonstrated the relative uselessness of newspapers as revealers (as opposed to disseminators).]

Blog Stats

  • 273,967 hits

OSWALD Newsletter

If you would like to receive OSWALD, a weekly open source news digest please send an email to oswald (with the subject "subscribe") at