Monthly Archives: August 2004

GoogleWatch Says ‘Google Is Dying’

Daniel Brandt argues that Google is dying: its index is failing to keep up with the growth of the web. And he thinks he knows why—Google hit the 4,294,967,296 limit on 4-byte ID numbers in C. (Perhaps coincidentally, perhaps not, Google claims to index more than 4 billion web pages.) If this is true, fixing it isn't trivial when you need to fix a large number of machines that are working in parallel

On sites with more than a few thousand pages, Google is not indexing anywhere from ten percent to seventy percent of the pages it knows about. These pages show up in Google's main index as a listing of the URL, which means that the Googlebot is aware of the page. But they do not show up as an indexed page. When the page is listed but not indexed, the only way to find it in a search is if your search terms hit on words in the URL itself. Even if they do hit, these listed pages rank so poorly compared to indexed pages, that they are almost invisible. This is true even though the listed pages still retain their usual PageRank.

…this became a problem that I first noticed in April 2003. That was the month when Google underwent a massive upheaval, which I describe in my Google is broken essay. When that essay was written two months after the upheaval, it would have been speculative to claim that the listed URL phenomenon was a symptom of the 4-byte docID problem described in the essay. It was too soon. But sixteen months later, the URL listings are beginning to look very widespread and very suspicious. It's a major fault in Google's index, it is getting worse, and it is much more than a mere temporary glitch.

Google is dying. It broke sixteen months ago and hasn't been fixed. It looks to me as if pages that have been noted by the crawler cannot be indexed until some other indexed page gives up its docID number. Now that Google is a public company, stockholders and analysts should require that Google give a full accounting of their indexing problems, and what they are doing to fix the situation.

If it turns out that google is missing huge quantities of stuff there will be a lot of angry IPO buyers. And I will have to change my one-stop-search habits.

Posted in Internet | 18 Comments

Power Blogging

You know it's a power blog when the author gets A Letter From the Editor—of the Economist no less.

Even though the Economist is going through a bad patch — predictable leaders, a US columnist who uncritically recites GOP spin points, and less coverage of out-of-the-way places like Albania than it used to have — that's still quite something.

Posted in Blogs | 1 Comment

Sign of Progress on CIA Torture

The signs have been clear from the start that the lion's share of the US's organized and systematized torture is by the civilians in the intelligence biz. In Iraq, their example, or pressure to emulate them, appears to have inspired those military torturers who were not simply free-lance sadists.

So far, though, it appeared that the CIA's conduct (and that of other similar agencies?) was out of bounds for a discussion which focused on the uniformed services. Perhaps, though, the ice is cracking.

C.I.A. Expands Its Inquiry Into Interrogation Tactics: Former intelligence officials say that lawyers from the C.I.A. and the Justice Department have been involved in intensive discussions in recent months to review the legal basis for some extreme tactics used at those secret centers, including “waterboarding,” in which a detainee is strapped down, dunked under water and made to believe that he might be drowned.

It has been known that, after the abuses at Abu Ghraib were disclosed, the Justice Department abandoned some legal opinions written in the months after the Sept. 11 attacks that had been used as the basis for the broad latitude allowed interrogators in using extreme procedures against suspected Qaeda detainees. In recent months, government lawyers said the legal opinions were too broad and were being rewritten to restrict the harshest interrogation measures.

The broader inspector general investigation into the agency's involvement in detention and intelligence in Iraq since May 2003 was ordered in May by George J. Tenet, who was then director of central intelligence. But additional questions about the C.I.A.'s practices center on a small number of high-level suspected Qaeda detainees being held by the agency outside Iraq in undisclosed locations around the world.

The C.I.A. has already scaled back some coercive methods used against detainees, although officials would not discuss specific techniques. Agency officials have demanded advance Justice Department approval for each tactic used against detainees and a new legal analysis of federal laws on the subject, including a statute that makes it a felony for American officials, including C.I.A. employees, to engage in torture.

One seminal document repudiated by the government was an August 2002 memo by the Justice Department. It concluded that interrogators could use extreme techniques on detainees in the effort to prevent terrorism.

Unfortunately, the NYT article also suggests that the CIA is seeking, or using, torture to question Khalid Shaikh Mohammed, a “high-level Al Qaeda suspect”.

Some of the most evil regimes have cloaked their vilest acts with a blizzard of paperwork and legality. Let's not end up like them.

Posted in Iraq Atrocities | Leave a comment

Heading Straight For Us

Hurricane Frances.

And they say it could be category four real soon.

Posted in Miami | 1 Comment

Not Just a River In Egypt

Rumsfeld in the NYT

“I have not seen anything thus far that says that the people abused were abused in the process of interrogating them or for interrogation purposes.” A transcript of the interview was posted on the Pentagon's Web site on Friday. Mr. Rumsfeld repeated the assertion a few hours later at a news conference in Phoenix, adding that “all of the press, all of the television thus far that tried to link the abuse that took place to interrogation techniques in Iraq has not yet been demonstrated.” After an aide slipped him a note during the news conference, however, Mr. Rumsfeld corrected himself, noting that an inquiry by three Army generals had, in fact, found “two or three” cases of abuse during interrogations or the interrogations process. In fact, however, the Army inquiry found that 13 of 44 instances of abuse involved interrogations or the interrogation process, an Army spokeswoman said.

OK, how do we explain this repeated lapse on the part of a supposedly hands-on detailed-oriented man?

  • Couldn't care less about torture
  • Deep denial, perhaps fueled by spread of 'war crimes trial' meme [I am not making this last part up]
  • Yes-men (and women) surrounding him never briefed him on the abuses at any time
  • Knows about abuses but ordered cover-up (why else send Sanchez to do intial report?) and no one dared tell him it failed
  • Is past his sell-by date

One slip of the tongue I could believe. But more than one, and so wrong on basic facts, about one of the most serious issues facing the Pentagon today?

Whatever it is, it's quite serious.

Posted in Iraq Atrocities | 3 Comments

Col. Brownback Update

During a fruitless search to find out the upshot of the motion to disqualify Col. Brownback from sitting in the first Guantanamo 'trial', I found this gem in the New Zealand Herald, regarding a byplay in the Salim Ahmed Hamdan proceeding:

Asked by the defence whether he believed the orders establishing the military commission were lawful, Colonel Brownback paused, and to the surprise of some observers, said: “I choose not to answer that question at this time.” Asked again by the military prosecutor, Commander Scott Lang, Colonel Brownback replied that he had “a duty to comply” with any order, even if it was “questionable”.

Does this mean anything, or just that he's learned to be cagey?

Posted in Guantanamo | 2 Comments

BBC Reporter Disdains US Counterparts — With Reason

According to CJR Campaign Desk, here's what Dusan Neumann, the BBC reporter assigned to cover the Cheney campaign, has to say about the BBC, the campaign, and US reporters:

Neumann, who grew up in Prague and who used a fake passport to defect to the U.S. in 1980, noted that the BBC proper doesn't seem interested in the election, since it's already apparently decided that it wants Kerry to win. By contrast, the press — and the public — in eastern Europe, view Bush more favorably, because the memory of totalitarianism is sufficiently recent that anyone who topples a dictator earns admiration.

As for the American news media, Neumann isn't impressed. Like many observers, especially foreign ones, he can't understand the obsession with trivia, and believes the press does a poor job at informing the public about the pressing issues of the day. He told me how he planned to begin his next written piece:

Whilst U.S. Marines, cavalry, Air Force and Iraq's security forces were tightening a noose around al-Sadr Mahdi militia and Grand Ayatollah Ali al-Sistani was hustled to Najaf, the cream of the national press core was counting apples, tomatoes, green peppers and ears of corn.

The last is a reference to this incident.

Posted in Politics: US | Leave a comment