NSA - Access It All
Some very damaging additional stuff about the NSA domestic spying will come up this week. A preview was given today on Face The Nation and on Meet The Press. The emphasis is now not on "collect it all" but the much more interesting question of how to "access it all". How does the NSA get the information out of the raw data.
@FaceTheNationSenator Udall says that all "phone records" are collected. But that is only half the beef. The NSA is collecting much more."The #NSA is literally collecting every phone record of every American every day...that is a violation of Americans' privacy" @MarkUdall
"Phone records" are the metadata of a call: Date/time of call, call length, originating number, location of originating number, destination number, destination location. If the implicated phones are mobiles additional information about the phone type and serial as well as location changes during the call may be included.
This metadata is useful to find connections between people, to reconstruct where they have been when and to find out about certain habits of the people involved.
But the content of the calls may be much more interesting.
As reckless and untruthful as the people at the head of the NSA have been proven to be there is absolutely no reason to believe that they do not also record the content of every call (and email and web access etc) of everything they could possibly get.
One internal document quotes the head of the NSA, Lieutenant General Keith Alexander, on a visit to Menwith Hill in June 2008, asking: "Why can't we collect all the signals all the time? Sounds like a good summer project for Menwith."Today 99% of call and internet traffic is transported through optical fiber cables. The NSA has access to every major fiber cable hub in the United States and in parts of Europe. It additionally taps into various undersea and land cables by clandestine means. It uses optical splitters that leave the original line working as before but copy the raw datastream onto an NSA line and feed it to some NSA datacenter where all will be recorded. As General Alexander planned five years ago the NSA it is by now really recording nearly all communication data.
But how can one use this data? How can one even access it? This is where the metadata comes in. Any name can be easily connected to a phone number and vice versa. Any IP address can be easily connected to a name. An IP address, a phone number, an email address, a name can then be used to automatically search through the recorded raw data streams to find and display the content data hidden in it. As Glenn Greenwald explained today on Meet The Press:
“The NSA has trillions of telephone calls and email in their databases. What these programs are are very simple screens, like the ones that supermarket clerks or shipping and receiving clerks use, where all an analyst has to do is enter an email address or an IP address, and it does two things: it searches that database and lets them listen to the calls or read the emails of everything that the NSA has stored, or look at the browsing histories or Google search terms that you’ve entered; and it also alerts them to any further activity that people connected to that email address or connected to that IP address do in the future. And it’s all done with no need to go to a court, with no need to even get supervisor approval on the part of the analyst.”Access to these search programs is not restricted to NSA personal. The NSA spends 70% of its budget on contractors. They do have, like Edward Snowden had, access to the search capability and thereby access to the meta- and content data.
Thinking further there is no reason to believe that these capabilities is restricted to certain facilities or just small circle of people. It is already known that U.S. and NATO soldiers in Iraq and Afghanistan had and have access to these systems and abused them. Does the State Department have access? Has the White House? Do political operatives have access? The very likely answers are "yes", "yes" and "yes".
The NSA claims that it can not search its own emails. I actually believe that to be somewhat true. The NSA task is to spy on others not on itself. Its internal email search capabilities may well be underdeveloped compared with its capabilities to search through the emails of others.
Likewise I doubt that its internal security is as developed as its external security. Trusted people with security clearance will have relative free access to its system (just ask Snowden) while any access from the outside will be heavily guarded.
So how much internal logging and controlling does the NSA have? Will every fishing through the accumulated data by trusted personal be recorded, logged and reviewed? I very much doubt this. Abuse then is likely to be widespread. Look up your nasty neighbor? Look up your former girlfriend? A political enemy? The urge to so will be great and the chance of getting rebuked over it will be small. Herein lies the mother of all scandals still to be unearthed.
Posted by b on July 28, 2013 at 16:45 UTC | Permalink
Here is a - German - visualisation of what you can do by connecting mobile meta data with internet information - it is based on the information of a German politician who got access to his stored mobile data.
And this here is the way it is applied
The New Zealand military received help from US spy agencies to monitor the phone calls of Kiwi journalist Jon Stephenson and his associates while he was in Afghanistan reporting on the war.Stephenson has described the revelation as a serious violation of his privacy, and the intrusion into New Zealand media freedom has been slammed as an abuse of human rights.
The spying came at a time when the New Zealand Defence Force was unhappy at Stephenson's reporting of its handling of Afghan prisoners and was trying to find out who was giving him confidential information.
The monitoring occurred in the second half of last year when Stephenson was working as Kabul correspondent for the US McClatchy news service and for various New Zealand news organisations.
The Sunday Star-Times has learned that New Zealand Defence Force personnel had copies of intercepted phone "metadata" for Stephenson, the type of intelligence publicised by US intelligence whistleblower Edward Snowden. The intelligence reports showed who Stephenson had phoned and then who those people had phoned, creating what the sources called a "tree" of the journalist's associates.
Posted by: somebody | Jul 28 2013 17:21 utc | 2
How long before corporations realize that competitors have access to their sensitive call contents? All it would take is to plant or pay off an analyst with an NSA contractor and--voila--your competitors' legal and patent strategies can all be overheard.
Want to know what your competitor knows in a lawsuit? Just ask your analyst to provide contents of calls between the competitor's management and its lawyers.
Only when the deep pocketed begin to realize that this snooping affects them will there be enough pressure to make Obama and his handlers in the NSA take stock.
Posted by: JohnH | Jul 28 2013 18:39 utc | 3
I simply don't believe the NSA can't search it's own internal (and external) mail. Every last byte of data that moves across the network is stored and logged automatically else it wouldn't function as a network. Retrieval is done in pretty much the same way the NSA retrieves data on us. They search it.
Posted by: William Bowles | Jul 28 2013 18:42 utc | 4
I would not at all be surprised if the lack of internal capabilities (eg searching internal emails) is intentional, or more accurately, there is more incentive to make access to the records of their process laborious or impossible. It certainly slows down/impedes public information requests. Or perhaps the system they do have to search it is subpar.
Posted by: Jessica S | Jul 28 2013 18:47 utc | 5
Did you see this James Bamford article? It doesn't tell you anything you shouldn't already know, but it does round up all the important points, especially the fact that the "metadata only" story is a transparent myth only repeated because it was in official statements from the start and that's what media do, repeat official statements as if they were facts. But the "metadata only" idea was demolished years ago. It was demolished by previous whistlebowers, long before it was officially put out as a cover story.
Posted by: Rowan Berkeley | Jul 28 2013 19:11 utc | 6
Reminds me of a snippet posted by b this week where NSA officials claimed they "didn't believe" Snowden tapped into the most sensitive NSA info. If they didn't believe, they either were lying or didn't know.
All of this is exactly why no such secrecy/massive info should exist. Trust nobody, nowhere near this much.
Posted by: Eureka Springs | Jul 28 2013 19:20 utc | 7
@JohnH#3:
Regarding use of corporate data, the US claims that its #1 threat of theft of corporate data comes from China, followed by Israel. Yet Israeli consultants and companies are enmeshed in America's cybersecurity programs. Consequently, Israel is well situated not only to obtain information about American businesses and patents that could give its industries an edge, it also is well-situated to obtain data about American business leaders and private information about their business dealings and personal lives (with all of that raw data, there's bound to be some valuable tidbits to use as leverage to encourage greater support of Israel).
Posted by: Rusty Pipes | Jul 28 2013 20:06 utc | 8
An essential part of understanding any government program is to understand the psychological grounding of the people involved, particularly when they are long-term, West Point-educated professional military officers like Lieutenant General Keith Alexander. These people have been regularly promoted through the system to high positions because they "get it." They never question the system and always do what they're told, while being experts at kissing up to their superiors. That last element is crucial.
Generally speaking, these are people who:
--believe that they have the training, the motivation and the opportunity to do what they believe is best for the country.
--having no experience in the private sector, their beliefs about what's best for the country are narrow and parochial, and oriented toward a strong government protecting helpless citizens whether they like it or not.
--doing what's best for the country might produce casualties of one form or another -- it's okay
--doing the right thing can only be done in secret, so nobody can mess with it. So classify everything, even if it's "for official use only."
--these people mutually support each other -- they all think alike.
So it helps to understand where these people are coming from. (I've been there, done that.)
Recommended reading-
Senator Ron Wyden of Oregon delivered a thoughtful speech on July 23 at a meeting held by the Center for American Progress Action Fund. Wyden’s speech was a summary of how the NSA has provided incorrect information to Congress and the public. He ended the speech warning about state tyranny:
James Madison, the father of our constitution, said that the the accumulation of executive, judicial and legislative powers into the hands of any faction is the very definition of tyranny. He then went on to assure the nation that the Constitution protected us from that fate. So, my question to you is: by allowing the executive to secretly follow a secret interpretation of the law under the supervision of a secret, nonadversarial court and occasional secret congressional hearings, how close are we coming to James Madison’s “very definition of tyranny”.
If the implicated phones are mobiles additional information about the phone type and serial as well as location changes during the call may be included.
Yup - " In fact, in a dataset where the location of an individual is specified hourly, and with a spatial resolution equal to that given by the carrier's antennas, four spatio-temporal points are enough to uniquely identify 95% of the individuals. "
More here - http://radamisto.blogspot.com/2013/06/i-dont-see-how-we-can-prevent-this.html
@Rusty "Regarding use of corporate data, the US claims that its #1 threat of theft of corporate data comes from China, followed by Israel."
And how much of what is pinned on China is actually Israel? I think it's an open question. Certainly hiding or altering ones IP address is hacking 101. I find it hard to believe that the PLA would continually leave their drivers licenses at the scene of the crime, so to speak.
"The report, in only relying on linking IP address to reach a conclusion the hacking attacks originated from China, lacks technical proof," the ministry wrote in its statement, "Everyone knows that the use of usurped IP addresses to carry out hacking attacks happens on an almost daily basis."
@Don It's nice to see Udall and Wyden finally open up on this. They've been given warnings for years now, but can open up now thanks to Snowden's information.
@JohnH - My guess is that we'll find that instead of providing info for one corporation to use against another (let's be frank, those top industries which are not dominated by near monopolies are firmly controlled by cartels) it will be shown that members of the security state will have used the pilfered information to generate sequester free profits from insider trading and the general manipulation of markets.
Posted by: guest77 | Jul 29 2013 0:44 utc | 12
b
To avoid misinterpretation: I'm by no means "picking" on you, quite the contrary, it's a compliment (and one that should be told openly).
Actually I perceive you as a man with lots of experience, not at last in reconnaisance. No matter how foggy the grounds and with red herrings around, you usually don't miss to reach a good understanding of the situation.
Sure enough, this is one of the reasons why many come and stay here.
That said, I sometimes see points where I feel I'd like some more details or point where I feel something escaped your view or plain and simple where my personal and professional experience puts me in a good position to carve out some issue somewhat more.
Do not take that as "picking on you", please, but rather as a compliment, as a situation where you've prepared fertile grounds and someone else takes on ;)
---
It uses optical splitters that leave the original line working as before but copy the raw datastream onto an NSA line and feed it to some NSA datacenter where all will be recorded.
Actually I think this "revelation" is politically motivated bullsh*t.
The line (the transmission line, cable) is the unknown, the one part outside the controlled environment of a data center. Accordingly (and for other reasons) the line is the one part that actually is the paranoically observed. Sure, it's feasible to install optical splitters somewhere but it's also an operation that is guaranteed to be discovered and to trigger alerts.
Let's look at the issue with a spies eyes: You can do it openly that is, with the carriers knowledge, you can do it half covertly or you can do it fully covertly.
The line splitter approach basically comes down to an open or, at most, half covert approach. Inserting that line splitter can't be done really covertly; those stories are, Pardon me, fairy tales.
And why go to that length? All one needs to do (being a government) is to demand certain equipment that, be it overtly or covertly, already has everything needed built in. Being at that, one wouldn't take the burden of grabbing data at the optical end, one would simply grab them at the electronic end which btw. typically has (for technical reasons) "mirror" capability built in anyway.
But there is another problem: If one really mirrored all data one would need the same capacity as the original cable. The problem is that, for instance, the combined capacity uk-zusa cross atlantic capacity is in the terrabits range - and lots and lots within that is more or less useless content (like video streams).
So, for both technical and tactical reasons one would - and almost certainly does - use another approach which is to preprocess data on location and to send "home" (to nsa) only those preprocessed data with maybe 1%-3% of the original bandwith (which is way better feasible).
But the problems go on. With each and every major international fiber ever layed they chose the - then - max available bandwith. For a simple reason. It's not the cable that is expensive but the laying.
Now one should remember what the limiting factor actually was, at least with fiber cables: it wasn't the cable, it was the electronics of the endpoint equipment. How much limiting was it? Well, it was difficult and demanding enough that even the best special processors (like network processors) were not good enough; they had to build special circuits ("ASIC").
And all that just to - "stupidly" - transfer the packets. Those packets were preprocessed and only had to be "packed" and put into the optical stream. *Any* further processing, no matter how minimal, would effectively cut down the bandwith severely.
So how does the nsa do it, what's the only sensible, feasible and manageable was to do it?
It's to have "preprocessors" that are specifically built (using ASICS again) and that do the job highly parallel (what a surprise! That happens to also be the was encrypted content is cracked). Those boxes first evaluate and then strip diverse protocoll layers (SONET/SDH, ATM, etc.), then they analyse the TCP/IP or phone headers, grab the metadata and decide whether the content is also grabbed ... then they repackage the whole stuff and finally send it over their own virtual channel/fiber.
What does this come down to?
- The carriers and governments in Europe and elsewhere *do* know. With rare exceptions where nsa really has to do it clandestinely, quite probably in less than 1% of all traffic, this all happens with the full knowledge of both carriers/providers and governments.
- Contents can be grabbed but usually are not. Grabbing the meta data only is not a political choice, it's a technical precondition; grabbing all data simply isn't feasible but in exceptional cases.
- The whole thing (almost certainly) works like 1 system and it has 3 layers.
The lowest layer is on location and does data mirroring ("grabbing") and very basic preprocessing and then sends the data to layer 2.
Layer 2 is regional centers with at least 1 per geo region and probably more. Here the preprocessed data is processed, probably mainly for triggers on a rather general and "generous" level, sorting out irrelevant data, and quite probably transposing headers into an internal format. An example for this might be a zusa base near, say, Manila (DeCIX). Also level 2 stations are probably manned 24/7 but only on a small scale.
The highest layer is layer 3. Those are the like of ne new nsa data center or menwith hill in uk. Here on layer 3 they have mid and long term storage, full analysis and more generally what one might call the "spplication" layer. At the same time those centers are the contact points for other agencies and it is here where requess, triggers and the like are fed into the system, and being propagated into layer 2 stations who again control layer 3 points.
Sorry for this lengthy post but I felt that it's about time someone looked at that question and tried to at least lay it out and roughly sketch the mechanism at work.
While I, of course, do not know any details, I have high confidence in my assessment of the sceleton because I happen to know the relevant technology quite well and, more importantly, its limits.
Posted by: Mr. Pragma | Jul 29 2013 0:58 utc | 13
Addition:
Sorry, I sometimes wrote "was" where it should be "way" (somehow I don't get accustomed to the local keyboards in my wifes country ... *g)
Furthermore, toward the end I put "DeCIX" after "Manila" which is plain wrong, of course. The reason is that I first had Frankfurt (with Decix) in mind but then chose Manila because Frankfurt imo either already is or is on its way to level 3).
And a third one: At the end of the next to last paragraph I wrote "propagated into layer 2 stations who again control layer 3 points" which, of course, should read "propagated into layer 2 stations who again control layer 1 points" (layer 1 being the lowest)
Apologies.
Posted by: Mr. Pragma | Jul 29 2013 1:08 utc | 14
The NSA claims that it can not search its own emails. I actually believe that to be somewhat true. The NSA task is to spy on others not on itself.
This is the most deliciously unsurprising NSA snooping revelation of all, so far. Anyone who hasn't realised what utterly incompetent self-deceivers the Yankees are hasn't been comparing the wishful thinking to the long-term outcomes.
Posted by: Hoarsewhisperer | Jul 29 2013 3:10 utc | 15
Great thread! Of course, the cartels will use the spooks' data to manipulate whatever they can. And Israel will do the same.
Now what's interesting is what the government "doesn't know." In particular, I'm thinking about the management and distribution of illegal drugs. Certainly with the vast array of electronic snooping tools at their fingertips, law enforcement could closely track every drug dealer in America and quickly learn who his friends and suppliers are, even if they had to get a warrant (which they probably wouldn't deign to do.)
Given this, it's truly mind boggling that law enforcement makes virtually no arrests of drug kingpins. This total "ineptitude" reinforces reports over the years by several intrepid reporters that the drug business is largely managed from inside rather than outside the government.
Posted by: JohnH | Jul 29 2013 3:14 utc | 16
@JohnH re: drugs - I couldn't agree more. It is hard to see how - in this day and age of surveillance and high technology and government largess - anyone could move that amount of goods and money completely undetected were the government serious about stopping it.
Of course the CIA has long been involved, and now we see the big banks paying "fines" for their laundering activities (of which I've read they net full HALF of the proceeds of such transactions). Clearly the government, by taking fines of less than what the profits amounted to, are just recouping part of their share.
And now we learn they've been tracking the most violent Mexican king pin - certainly responsible for many thousands of deaths and god knows how many hundreds of tons of product he moved over that time - for years without making an arrest. Absurd.
Posted by: guest77 | Jul 29 2013 3:33 utc | 17
Greedy, stupid, Yankee supremacists like to compare their Podunk 'empire' to the Roman Empire. But that's silly. The Romans brought the notion of state of the art infrastructure, and education, to many of the places they conquered.
All the Yankees have brought is death, destruction, looting, piles of rubble, and crass ignorance. And their 'Homeland' is starting to look the same.
Posted by: Hoarsewhisperer | Jul 29 2013 3:52 utc | 18
Guest77 @17: tracking Mexico's most violent drug kingpin for years...guess he was with us before he was against us...like Saddam, Torrijos and countless others. It would be interested to know who Omar Torrijos and Miguel Angel Treviño crossed.
Posted by: JohnH | Jul 29 2013 4:01 utc | 19
Here's an actual example of one of the many uses of the NSA/corporate info gathering:
http://therealnews.com/t2/component/hwdvideoshare/viewvideo/76499
Posted by: ben | Jul 29 2013 5:05 utc | 20
Tom Burghardt as usual following all the popular tech mags very closely:
http://antifascist-calling.blogspot.co.uk/2013/07/big-data-dynamo-how-giant-tech-firms.html
Posted by: Rowan Berkeley | Jul 29 2013 7:30 utc | 21
JohnH @19: read John Perkins' "Confessions of an Economic Hit Man" for some background on, and motives for, the death of Torrijos. Given the methodologies described by Perkins, I suspect Trevino ceased being useful to the deep state and was taken out of circulation.
Posted by: Yonatan | Jul 29 2013 10:11 utc | 22
@22
I'm not entirely sure he's been taken out of circulation. To see him strolling with no handcuffs and without a real presentation like every other thug gets from the Mexican Navy... perhaps he's still running the show for the time being.
Posted by: guest77 | Jul 29 2013 10:40 utc | 23
Yonatan @22. You're right. I wrote Torrijos but was thinking Manuel Noriega, who was a useful drug runner...until he wasn't.
Posted by: JohnH | Jul 29 2013 14:00 utc | 24
Trouble for Merkel?
news report:
German public opinion is scandalized over recent revelations in National Security Agency documents leaked to the media by Edward Snowden, the former NSA analyst who fled to Russia. According to the allegedly top secret NSA documents, German intelligence services under Chancellor Angela Merkel collaborated with the NSA to perform massive data mining on ordinary German citizens. Amid charges of gross negligence or willful betrayal of Germany, Merkel, who has been a reliable ally of the United States, faces worsening prospects to remain head of the ruling center-right Christian Democrats and chancellor in the general elections on Sept. 22.
Don Bacon @25
This is certainly as it should be, but last I read (probably here) the word was that the surveillance disclosures would probably have little impact on Merkel's chances in the upcoming general elections.
The provenance of your "news release"
(http://www.lignet.com/ArticleAnalysis/Germany-Snowden-s-NSA-Leaks-Undermining-Merkel-G)
is interesting, as LIGNET is not any ordinary news source.
obob
Posted by: oboblomov | Jul 29 2013 18:57 utc | 26
how come ,ok,how come there are no street protests,no marches to the statue of whatchamacallit.where is the outrage mr American citizen,take your ear phones off,unplug your little worm from the machine and do something to protest -how about this week,first sunday of august we all take a shoe and throw it at the guy in charge of the nsa machine.
Posted by: jub | Jul 29 2013 22:00 utc | 27
It's all tied together... war on journalist's war on truth seekers..
However, no none is talking about the other whistle-blower...
Manning Verdict May Topple Journalistic Paradigm
Court artist's rendition of the Bradley Manning trial. (Screen grab via Democracy Now!)
After more than three years in prison, 10 and a half months of which were in solitary confinement, enduring cruel and degrading treatment throughout his incarceration, Pfc. Bradley Manning now awaits a final verdict in his court-martial. The verdict will be announced Tuesday at 1 p.m. EST, according to a spokesperson for the military district of Washington.
Government prosecutors called Manning an "anarchist" and "traitor" last Thursday during closing arguments at Fort Meade, Maryland, where Manning's court-martial has been playing out for nearly two months now. But the prosecution's character assassination of Manning is not the worst of it.
Also see, Espionage: Now, with No Damage Envisioned
A recently unsealed decision from Colleen Kollar-Kotelly just changed the interpretation of the Espionage Act for Washington DC to cover leaks that wouldn’t even harm the US.Judge Colleen Kollar-Kotelly ruled that the prosecution in the pending case of former State Department contractor Stephen Kim need not show that the information he allegedly leaked could damage U.S. national security or benefit a foreign power, even potentially. Her opinion was a departure from a 30 year old ruling in the case of U.S. v. Morison, which held that the government must show that the leak was potentially damaging to the U.S. or beneficial to an adversary. (In that case, Samuel L. Morison was convicted of unauthorized disclosure of classified intelligence satellite photographs, which he provided to Jane’s Defence Weekly. He was later pardoned by President Clinton.)
“The Court declines to adopt the Morison court’s construction of information relating to the ‘national defense’ insofar as it requires the Government to show that disclosure of the information would be potentially damaging to the United States or useful to an enemy of the United States,” Judge Kollar-Kotelly wrote in a May 30 opinion. The opinion was redacted and unsealed (in partially illegible form) last week.
Posted by: Uncle $cam | Jul 30 2013 1:03 utc | 28
@ #28
How can anyone prove that information could or could not be beneficial to an adversary? So I can understand the judge's ruling.
Which is another way of saying that information doesn't take sides, and so all information should be free and accessible.
No-one is talking about the other whistle-blower... Manning Verdict May Topple Journalistic Paradigm. Posted by: Uncle $cam | Jul 29, 2013 9:03:41 PM | 28That's not true. People who think Snowden is phony have created a meme whereby Snowden is an attempt to distract attention from Manning, and is succeeding in this. You can do story counts in the MSM if you want, but we aren't the MSM, and we're perfectly capable of keeping track of Snowden and Manning simultaneously. An interesting story about how the Manning indictment has actually been altered retrospectively by Military Judge Denise Lind, here
Posted by: Rowan Berkeley | Jul 30 2013 6:53 utc | 31
"He stressed that computer technology experts have not discovered anything new about Washington’s scope of cyber espionage. Instead, the whistleblower only reinforced the belief that Russia needs to strengthen its cyber security, Rogozin said."
Rogozin is Russia's deputy PM.
His remarks come a day after MI5/6 have claimed that certain Chinese made computers are engineered to allow hackers to access their contents.
Rogozin says:
“This is the truth that pushes us to act and hastens the creation of our own base of electronic components which we cannot do without,”
The era of Free trade is coming quickly to a close. Even as monster FT deals such as the Trans Pacific and EU-NAFTA are being finalised, the tariff (or rather non-tariff) barriers are going up again.
The problem with the international division of trade ('We'll make beads, you make slave cargoes; We'll make cars. You make turnips.') is not just that it is skewed to favour the west/north but that it underdevelops the underdeveloped countries, pushing them further into dependency.
Nationalism, political and economic, is part of the alternative to the phony internationalism of US hegemony. As Iran shows nations wishing to rule themselves have to develop their own resources, technology, arms and food supply in order to save themselves from foreign dictatorship.
Posted by: bevin | Jul 30 2013 12:43 utc | 32
Yeah, I saw that too. I have two Lenovo machines, and the last of my worries is that the PRC will hack into them.
Posted by: Rowan Berkeley | Jul 30 2013 13:23 utc | 33
Rogosin is right and he is addressing a painful oversight of Russia.
While Russia generally has excellent capabilities in science and technology there is a painful lack concerning complex, in particular digital semiconductor devices.
China has since quite a while now been using an interesting and smart approach. They widely ignored the currently most used (in PCs) Intel/ADM x86 technology and instead bet on MIPS. While not being that widely known and recognized MIPS is a very interesting and proven concept and offers the additional benefit of being inherently simpler in many ways while offering the potential to very high performance. The current generation offers performance that is similar to modern Pentium generations while using considerably less energy. Actually the Chinese are currently working on a supercomputer based on those chips.
Of course, it is of major importance and help that China has mastered semiconductor production quite early and is nowadays at above the level of 90% of the western companies (but not yet capable to reach the very top).
Russias problem is very serious in that Russia basically is quite dependent on either western or Chinese chip supplies. Furthermore, the design and production of complex modern semiconductors is rather complicated and not easily reachable. While Russia can for some time rely on Chinese or even western supplies, it is of utmost importance for any modern major power to have their own independent capabilities. To reach these Russia will, if sailing with full power and spending billions, need at least 10 and quite probably 20 years.
Posted by: Mr. Pragma | Jul 30 2013 15:33 utc | 34
Posted by: Hoarsewhisperer | Jul 28, 2013 11:52:26 PM | 18
yes unlike the americans
life of brian
http://www.youtube.com/watch?v=ExWfh6sGyso
Posted by: brian | Jul 30 2013 16:44 utc | 35
Some observers noted, at the time that the Stuxnet affair was being revealed to the public, that by engaging so closely in such activities firms such as Siemens were demonstrating to potential customers the real risks of dealing with corporations dependent on the US government.
The short term gain of sanctions and sabotage has to be weighed against the long term pain of lost markets, suspicious customers and. most importantly, future competition from firms capitalised and subsidised by states determined to become independent of the US-Japan-Israel-NATO axis.
Posted by: bevin | Jul 30 2013 16:54 utc | 36
The ongoing E NSA spying scandal is truly fascinating. The situation obviously shows the unbelievable issues the US has regarding privacy.But it’s not only our president. Even private Americans are now snooping on each other. Without knowing any thing about you; non-techies,Marsha,your nextdoor neighbor,Sammy can use web based resources like IntelliGator and learn your criminal records, address, employment history...etc. And exactly like the NSA - it’s 100% legal.Shady but nonetheless legal.
This must change. In the digital era, there’s too much info out there. And govt cyber-terrorist are getting more innovative every day. When will it end?
The US Government domestic spying story is really troubling. But seriously -- what did you expect. First off The movie industry has already prepared Americans for this type of spying. We all applaud as the movie police breaks into telephone records to catch the "bad" guys. And despite not having a warrant, we have no trouble with it. But we act surprised when we learn the same technology exists and is being used on us.
What's more - You don’t have to be a big government to spy. Businesses monitor us all the time. Anyone with a internet connection can look into your history from Google background searches. On-line spying tools are making traditional private detectives redundant.
It’s all too legal - and all very frightful.
Posted by: Tanmoy Misra | Jul 30 2013 17:12 utc | 37
@Mr. Pragma @13 - I concur with you. I of course could have explained the finesse of Sonet/SDH decapsulating and layer 2 and 3 IP sniffing etc. I am pretty knowledgeable on those levels and well versed in using wireshark. But I try not to go too deep in my blog posts. The optical splitter is a stand in for those details. (Look up the Jimmy Carter submarine which was build to put optical splitters on under water cables.)
But I still believe that the NSA pretty much has a "raw duplicate" of the traffic. yes, it will cut out the netflix and youtube and bittorrent content from what it aggregates in an early stage and only keep the metadata for those but phone traffic etc. will be transferred in full. As for cable capacity - the NSA has practically an unlimited budget. To dig in new cables should it need them to haul the data to Utah is not a problem.
Ron Wyden: Intel violations ‘troubling’
Sen. Ron Wyden said Tuesday that U.S. intelligence agencies’ violations of court orders on surveillance of Americans is worse than the government is letting on.
...
“We had a big development last Friday when Gen. [James] Clapper, the head of the intelligence agencies, admitted that the community had violated these court orders on phone record collection, and I’ll tell your viewers that those violations are significantly more troubling than the government has stated,” Wyden said.
@b -comment 38
Nah, they don't need to dig their own cable. They just buy or lease some of the spare capacity on existing ones. A couple of years ago NTT was installing a new fat cross-pacific cable. I remember reading that only 50~55% would actually be lit up (activated). And of that active capacity, less than half would be used for general traffic, the rest was/is reserved for big corporations – e.g the big stock-market manipulators or Google, which leases spare fiber capacity to sync their various data centres across the globe. I suspect NSA and their friends just set up some small ‘data-management’ corporation to lease some of that capacity for their own use.
PS- something Mr Pragma didn't address directly in comment 13: backdoors in network equipment (core routers and switches). Remember the US ‘outcry’ when Chinese corporations (Huawei) were the main back contenders for installing a new fat pipe between Australia/NZ and continental Asia…
Posted by: Philippe | Jul 30 2013 23:45 utc | 40
b (38) & Philippe (40)
Let's play a bit (haha) with numbers:
While it's not reasonably feasible to know the real with any reasonable level of precision, one can make an educated guess with a strong tendency toward the low end and say that the combined capacity currently used is well above 20Tb.
In other words, the amount of data transferred is >= 2TB (that is 2.000 GigaByte) every second. This just nicely matches currently higher end disks.
That's 86.400 of those disk per day or, using high density disk storage systems or around 200 data center racks full of disks or, in terms of densely packed data center space 180 m² net.
According to the information available the new nsa datacenter in utah will provide ca. 100.000 ft² ~ 9.300 m² space (for the datacenter. There is lots mre for administration et al.) or ca. 51 days worth of global data.
However, sure enough this data center will not hold data storage only, so it gets closer to 1 month worth of global data. This calculation happens to roughly match the available information of "some exabytes of data".
But this data must also be evaluated, analysed and processed to have any intelligence value. Doing that with ca. 170 PB (Petabytes ~ 1.000.000 Gigabytes) that currently no known system is capable of doing even on a rather low level.
At the same time this approach would transport data of which in between 70% and 90% are completely worthless (movies, music, google content (not requests!) etc.)
Sure, assuming nsa has an unlimited budget (which I doubt) they could just lay new cables (yes cable*s*, plural, because one would, of course, lay 2 cables with a solid geographical distance) across the globe for quite some billions of dollars. But why would they do that? Just to stubbornly transmit every bit available although by far most of that would be worthless junk? That wouldn't solve any problem for them but it would cost huge money and, possibly more importantly, time without gaining anything relevant.
So I don't see that happen.
Any not completely nonsensical approach would filter the data at least somewhat on location and then send the part that is not definitely irrelevant data to nsa. This not only would very considerably decrease the vital bandwidth needed for nsa but it would also allow to store way more relevant data and to process them.
Looking at the numbers involved, at sound technical proceedings and at the reality on ground I'm quite confident that my understanding is not too far from what is actually happening and how.
And yes, I know about the Jimmy Carter (USS 23 if I'm not mistaken) but I also know physics, optical transmission and that pretty everyone takes the optical splice as rumour. What is *known* is that zusa actually did eavesdrop on Sowjet military cables, copper cables, that is.
But no matter, whether they could so it, the question is "why would they"? From what we know, most carriers happily co-operate since years and have no qualms about it. So, why we assume that nsa takes extremely costly and complex maneuvers upon them to get what they could get cheaply and comfortably within a well established framework of willing and cooperating carriers at the landing points? And, just btw, in order to be useful that optical splitter also needed to be connected to a new fiber cable laid by the Jimmy Carter submarine which again would need to be landed anyway.
Philipe is right, there are many, probably well founded, rumours about backdoors in core routers and other backbone equipment. This, however, is on a rather different level (technically) and therefore somewhat less useful (for the purpose of what nsa wants).
I think btw. that the "Huawei - they have mined our infrastructure with backdoors!" cajole is based not only on - almost certainly well founded - zusa fear but also imo pro-actively blown up as precaution because, you bet on it that nsa has done exactly that themselves. Let me put it this way: If I had sensible data to transmit I wouldn't use cisco equipment (or nortel or ...). This is no fun; I actually did use different algorithms on different architectures for really sensible data.
But this opens another can of worms, namely the question how one might protect onself or a client on a professional level from cia, nsa and the like. So I'll rest my case for the moment.
Posted by: Mr. Pragma | Jul 31 2013 1:15 utc | 41
More from Snowden:
Revealed: NSA tool that collects 'nearly everything a user does on the internet'
• XKeyscore gives 'widest-reaching' collection of online data
• NSA analysts require no prior authorization for searches
• Sweeps up emails, social media activity and browsing history
training materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search. The request is not reviewed by a court or any NSA personnel before it is processed.XKeyscore, the documents boast, is the NSA's "widest reaching" system developing intelligence from computer networks – what the agency calls Digital Network Intelligence (DNI). One presentation claims the program covers "nearly everything a typical user does on the internet", including the content of emails, websites visited and searches, as well as their metadata.
Analysts can also use XKeyscore and other NSA systems to obtain ongoing "real-time" interception of an individual's internet activity.
...
Anecdotes.
CH > Since about 5 years exists a National Data Base that collects all prescription meds. orders, the pharmacy employee told me. When asked who could consult it, she answered, well, the .. your doctor, and anyone in the pharmacy, as we are here to serve you!
Nothing was ever said about it in the press or afaik on TV and ppl aren’t aware it. I asked about 20 ppl, they only appreciated that their pharmacy knew what they needed and ordered the meds in time and communicated with their health insurance and so on, easier billing, paperwork, etc. Basically, these records are semi - public as anyone who is curious and determined can access them easily ..and that is your med history laid out right there.
From a public health pov (or a corporate drug co. pov) this information is prime. You can track over-subscribers, count the cost of X antibiotic and think about replacing it with generic ABC, calculate how much the insurance is paying for junk meds (which ones to ban), and so on. Potentially, it may lower the cost of health insurance, improve prescription practices, even patient compliance, etc. Good stuff ? Implemented without any public discussion whatsoever.
It is information creep.
US > A huge number of divorces are based on or due to evidence of infidelity (incl. what is called ‘emotional cheating’, just intimacy...) garnered from Facebook, emails, phone records, sms, either accessed by accident or found by ‘snooping‘ which is often considered legit if one has ‘doubts, suspicions..’ What used to be face-to-face flirting (or more), easily hidden, washed over, or denied, evanescent as to precise details, and immaterial, or successfully held secret, is now stark in black and some pastel color. Men track their women (phone records, GPS data, etc.), women look to see passwords typed in and access phone, chat logs, other, to discover He said --- Hun I love Your Xx --- to some cheap chick... Ka-Boom! Family destroyed, toddlers wailing.
CH > Was member of the board of a large U library when the point was reached that all books borrowed in public U libraries in the country could be tracked. Now, libraries are very concerned with privacy but in CH nobody would dream of using book-reading as ‘evidence’ for ‘terrorism/other’ so that was never a concern, not even an unspoken one. These libraries are desperate about book theft and book defacement. One can easily imagine what could have been done....but it wasn’t. Chinese walls were set up, the info was hyper protected, the higher level agreed - yet the info would potentially have served to catch criminals, reduce losses to libraries which equals more text books for students etc. So from the outside it was a hard call. and probably a rare ex. of self-denying privacy protection.... This is an example of, where to draw the line?
(I am totally against NSA spying and am not justifying or excusing it in any way, nor making light, just describing...)
Posted by: Noirette | Jul 31 2013 13:11 utc | 43
@Mr.Pragma Here are some numbers from the new Guardian story. These numbers are from 2008! Todays numbers will be much bigger.
The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."To solve this problem, the NSA has created a multi-tiered system that allows analysts to store "interesting" content in other databases, such as one named Pinwale which can store material for up to five years.
It is the databases of XKeyscore, one document shows, that now contain the greatest amount of communications data collected by the NSA.
Here is the full XKeyScore Presentation
It says on page 2: "Stores full-take data at collection site ..."
I've made a pdf of the 32-page Guardian presentation of X-Keyscore:
http://niqnaq.files.wordpress.com/2013/07/x-keyscore.pdf
Posted by: Rowan Berkeley | Jul 31 2013 14:45 utc | 45
That 32-page presentation is not the source of some of the frames in Greenwald's accompanying article, even some marked with the "X-Keyscore" logo come from another X-Keyscore presentation, or from different presentations altogether. As I pointed out earlier, the first WaPo article about the NSA leak said there were a total of 91 slides, "plus supporting documentation".
Posted by: Rowan Berkeley | Jul 31 2013 16:40 utc | 46
It looks as if I was pretty close to the reality. According to now published material, there are around 150 "points" around the world where nsa gathers and - at least to a degree - processes grabbed data.
What they (in the general press) called "points" is what I called "level 2" that is, (not large) intermediate stations where regionally grabbed data are concentrated, processed (on a low level) and raw-filtered before being sent to the main nsa centers (e.g. europe -> uk spycenter).
Posted by: Mr. Pragma | Jul 31 2013 21:47 utc | 47
The comments to this entry are closed.
"Herein lies the mother of all scandals still to be unearthed."
The mother of all scandals has already been 'identified'... It is the 'ignorant' general public. The US has had its imperial/fascist tendencies for a very good while. It was 'We The People' who were maintaining the checks and balances. The last C&B may have been the removal of that other crook president. Unfortunately, today's 'general' public has been thoroughly demonized and assimilated. They are only interested in entertainment, be it a murder trial, a presidential election, or shopping at walmart. The real question may be; Are the general public in EU assimilated too???? Are they awake? If they are, they are not put on the leash on their subservient governments.
Posted by: Rd. | Jul 28 2013 17:10 utc | 1