Calculating the "5%"

The most difficult thing for an economic theorist, or a scientist of any stripe, is making data of sufficient quality when it does not exist for the "taking."

For example, say you wanted to study trees but only had data on forests. How do you find out information on the trees? Worse, your theory demands the historical tree data too (hat tip, to the late John Dales: see separate blog post, on an unrecognized Canadian *star*).

For us, in the post "When 5% is More Than Half", the work underlying the post compared the census industry data with the sampling data, that ought to have shown similar results to be relevant.

It did not and this is not for us to explain but the authors of the survey. They have only partially explained their results, but we attempted to an explanation that others may not share.

To the work performed, we had to make some assumptions from knowing well the prior industry census that attempted to cover every company that produced new releases.

The new survey was not performed in the same fashion: a survey of the 95% of industry by financial revenues, sufficient to satisfy a high confidence level on sample statistics that could be applied to project this group's population level data. It is very important to note, that the aim of of the methods employed was not to estimate the 100%: it was only to estimate sufficiently that 95%.

For outsiders, they do not tell you the "N" (population) or "n" (sample) used. The number of companies in the "95%", or the number of companies surveyed to project the "N" of the 95% group, the little "n", is an unknown. It would also have little meaning as the industry definition was updated from activity based to the NAICS classification system, that included music publishers, sound recording studios and one other type of firm not relevant to the creation of new releases.

For outsiders, they did though tell the "N" for the "new release" industry in the census data for the overlapping periods, 2000 and 2003. You can see those numbers compiled from the census in the table near the end of the post "When 5% is More Than Half."

Further, between company types in the census data, the number of foreign controlled Canadian companies and Canadian controlled Canadian companies were disclosed. As well, the relative industry shares of revenues were also disclosed or discernible. 84.7% v. 15.3%, foreign controlled v. Canadian controlled on total industry revenues, in 2003 (13% v. 87% in 2000).

Oddly or not, the 95% share claimed captured by Statistics Canada in the survey, closely resembled the claims of the then CRIA in 2005 who at the time had Canadian controlled companies. The Canadian controlled firms however subsequently defected from the CRIA in 2006. Calculating the 5% unsampled and unreported, by differencing the sample data from census data, left it an arithmetic exercise to calculate the remaining 10.3% of Canadian controlled company revenues, from the 2003 census data. The balance is quite logically the CRIA of today, sans Canadian controlled companies who are in the business of putting new Canadian artists releases into the market.

To do this required an assumption however: there are no remaining companies in the CRIA that are Canadian controlled. This is not a known fact as the CRIA does not disclose its membership currently on its public website, as a private industry association. They know and that is who knows. Now.

One can though suspect strongly that no Canadian controlled firms remain that also produce new Canadian artist releases. When the defections occurred in 2006, it was clear with the Canadian interests lay in expressing greater Canadian content requirements to the CRTC, were not shared with the "A" members of the CRIA, the members who can vote, who in 2003 had a 2.7% of their new artist releases Canadian in nature. The question is though did any Canadian controlled companies remain who create new Canadian artist releases? Its doubtful both by the views expressed by the CRIA, and the known prominent members of the CRIA who defected in 2006.

How did we know who was in the CRIA? We looked using the "Way Back Machine" at Archive.org in the U.S. and the sometimes better, Bibalex, from the Great Library in Alexandria, Egypt. In the Web 2.o world, if you are interested in something, very often the information that is not available, might have been at one time.

A segue on this latter issue: An interesting article appeared in the Globe & Mail last week on correcting one's reputation on the Internet, "Scouring Away Your Digital Dirt". It totally missed the issue of the archiving of the Internet that is an known active process by several parties. It also missed the archiving that is also a likely action by unknown parties, including interested business entities for competitive intelligence. Perhaps Reputation Defender does have as part of its services the removal of archived information from at least the known archivers. It was unstated in the article. From the unknown archivers, including interested individuals, all we can say is good luck! The information on the Internet is a postcard that everyone gets to look at, to save, to print, and to use, fairly or not so fairly. All who post information should never forget this! Once up, its forever, somewhere. Yes you can take down a publication of some information for present and future dissemination. But the past is also present today on the Internet as well.