1404 1 UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLUMBIA 2 3 ELOUISE PEPION COBELL, : Civil Action 96-1285 et al. : 4 Plaintiffs : : Washington, D.C. 5 v. : Monday, October 22, 2007 : 6 DIRK KEMPTHORNE, Secretary : of the Interior, et al. : 7 : Defendants : AFTERNOON SESSION 8 TRANSCRIPT OF EVIDENTIARY HEARING 9 DAY 7 BEFORE THE HONORABLE JAMES ROBERTSON 10 UNITED STATES DISTRICT JUDGE 11 APPEARANCES: 12 For the Plaintiffs: DENNIS GINGOLD, ESQUIRE LAW OFFICES OF DENNIS GINGOLD 13 607 14th Street, NW Ninth Floor 14 Washington, DC 20005 (202) 824-1448 15 ELLIOTT H. LEVITAS, ESQUIRE 16 WILLIAM E. DORRIS, ESQUIRE KILPATRICK STOCKTON, L.L.P. 17 1100 Peachtree Street Suite 2800 18 Atlanta, Georgia 30309-4530 (404) 815-6450 19 KEITH HARPER, ESQUIRE 20 JUSTIN GUILDER, ESQUIRE KILPATRICK STOCKTON, L.L.P. 21 607 14th Street, N.W., Suite 900 Washington, D.C. 20005 22 (202) 585-0053 23 DAVID C. SMITH, ESQUIRE DANIEL R. TAYLOR, JR. ESQUIRE 24 KILPATRICK STOCKTON, L.L.P. 1001 West Fourth Street 25 Winston-Salem, North Carolina 27101 (336) 607-7392 26 27 Bryan A. Wayne, RPR, CRR Official Court Reporter 1405 1 For the Defendants: ROBERT E. KIRSCHMAN, JR., ESQUIRE JOHN WARSHAWSKY, ESQUIRE 2 MICHAEL QUINN, ESQUIRE J. CHRISTOPHER KOHN, ESQUIRE 3 JOHN KRESSE, ESQUIRE U.S. Department of Justice 4 1100 L Street, N.W. Washington, D.C. 20005 5 (202) 307-0010 6 JOHN STEMPLEWICZ, ESQUIRE Senior Trial Attorney 7 U.S. Department of Justice Commercial Litigation Branch 8 Civil Division Ben Franklin Station 9 P.O. Box 975 Washington, D.C. 20044 10 (202) 307-1104 11 Court Reporter: Bryan A. Wayne, RPR, CRR Official Court Reporter 12 U.S. Courthouse, Room 6714 333 Constitution Avenue, NW 13 Washington, D.C. 20001 (202) 354-3186 14 15 16 17 18 19 20 Proceedings reported by machine shorthand, transcript produced 21 by computer-aided transcription. 22 23 24 Bryan A. Wayne, RPR, CRR 25 Official Court Reporter 1406 1 P R O C E E D I N G S 2 THE COURT: All right, sir. 3 MR. GINGOLD: Good afternoon, Your Honor. 4 THE COURT: Good afternoon. 5 (The witness resumed the stand.) 6 DIRECT EXAMINATION (Continued) 7 BY MR. GINGOLD: 8 Q. Good afternoon, Mr. Duncan. 9 A. Good afternoon. 10 Q. Mr. Duncan, this morning you testified with respect to two 11 of the problems you identified at the outset of your testimony: 12 The missing data problem and the sample selection problem; is 13 that correct? 14 A. Correct. 15 Q. There are three other problems which you identified which 16 you characterize as major. Is that a fair statement? 17 A. Yes. 18 Q. The next one is irrelevant sample design, correct? 19 A. Correct. 20 Q. What do you mean by an irrelevant sample design? 21 A. Here the notion that is captured in that concept is what 22 was the sample designed, what question was the sample designed 23 to answer. And what we're able to observe from looking at the 24 administrative record, and specifically the NORC reports in the 25 administrative record, is that the sample design was really -- 1407 1 and the LSA, the sample that was drawn, was, as its name 2 implies, really a litigation support activity or an accounting, 3 as I believe this -- litigation support accounting. 4 But it was really designed to estimate the dollar exposure 5 consistent with the kinds of things that I would do in a damage 6 case, in preparation for a mediation or a settlement discussion, 7 trying to estimate the overall exposure. And it wasn't really 8 designed to answer specifically information about the accuracy 9 and completeness of the individual transaction listing and 10 account balances, and the information that was provided inside 11 of the LSA reports really went to those gross dollar questions, 12 not corrections that would potentially apply to individual 13 accounts. 14 Q. Now, you're making this statement based on your experience 15 with regard to damages issues, aren't you? 16 A. Yes, that's correct. 17 Q. And you've done that in the course of your career; you've 18 provided expertise in that regard, correct? 19 A. In hundreds and hundreds of cases. 20 Q. So what you've seen in the NORC sample is not materially 21 different from what you yourself have done in trying to estimate 22 liability, correct? 23 A. That's correct. When that's the question on the table, 24 this would be an appropriate activity to do. 25 Q. And that's a sound approach if you're a defendant, isn't 1408 1 it, in a case, correct? If you were defending yourself? 2 A. To try to come up with some estimation, it's helpful in 3 trying to negotiate your way into, is there going to be a 4 mediation, it's helpful in trying to develop your trial 5 strategy, to have some idea of the rough magnitude. 6 Q. And that is not your understanding, however, as to whether 7 or not it is an accounting that discharges a fiduciary duty, 8 correct? 9 A. It would be inconsistent with that notion in my mind. 10 Q. Now, could you give me some examples of transactions that 11 you're aware of where accounts are misstated? Do you have any 12 idea of any, or do you have any in mind? 13 A. Of specific accounts that are misstated? 14 Q. Yes. With respect to overpayment or underpayment. 15 A. Well, the only specific information we've been provided as 16 part of the administrative record was for the Eastern Region. 17 Q. And the Eastern Region tells you what based on your 18 recollection? 19 A. Well, there are specific instances in the Eastern Region 20 analytical work done by Deloitte & Touche that show there are 21 dollar discrepancies, there are dollar differences between the 22 amount that is in the IRMS, in the electronic ledger era that 23 was sampled, and when they were reconciled back to whatever they 24 were reconciled against. There were dollar differences noted. 25 Q. Let's talk about the historical accounting project itself. 1409 1 I'll call up for purposes of our discussion Bates 33-3-14. 2 MR. GINGOLD: Your Honor, the historical -- 3 THE COURT: Bates 33-3-14? 4 MR. GINGOLD: That's correct. 5 THE COURT: Okay. 6 MR. GINGOLD: That's part 2 of what is the historical 7 accounting project, and it has -- the historical accounting 8 project has two basic Bates stamp numbers 33-1, 33-2, and 33-3. 9 Part 2 is 33-3, and the page that I'd like to discuss with 10 Mr. Duncan is 33-3-14. 11 BY MR. GINGOLD: 12 Q. Mr. Duncan, you've read this before, haven't you? 13 A. Yes, sir, I have. 14 Q. That's right. And when we're dealing -- here we're dealing 15 with sample debit transactions and sample credit transactions, 16 correct? 17 A. Correct. 18 Q. And under sample debit transactions, the statement is 19 "There were no differences observed for debit transactions," and 20 then it goes on, "therefore, there were no underpayments or 21 overpayments." 22 What is your understanding of those two bullet points? 23 A. This gets interwoven with a lot of the topics we've talked 24 about this morning, which is first of all what is an error. So 25 what they're saying here is as defined by the ASM, they did not 1410 1 identify, of the transactions that they reconciled, using the 2 different levels of reconciliation, of those that they 3 reconciled were there any differences that were observed. 4 And the other overlay here is it's not just any difference; 5 it has to be above their de minimis criteria. So there had to 6 be a difference in the transaction of at least a dollar. So 7 what they're saying here is actually there are no differences 8 observed that were below our de minimis threshold, for the ones 9 that were reconciled using the definition of reconciliation 10 inside the ASM. 11 Q. And you're going to be discussing this later in your 12 testimony, but the de minimis, a dollar, is that so 13 insignificant it's not worth doing any work on? 14 A. It matters for two major reasons. The first one is when 15 you're doing sampling, you're going to try to extrapolate those 16 results up to a much broader population, so you may be sampling 17 a small fraction of the transactions. So anything that you 18 discover is amplified many times over when it's extrapolated to 19 the population. 20 So how -- and remember the attribute sampling, the yes/no, 21 deciding what is yes and what is no, that makes a difference. 22 Even if it's only a dollar, it makes a difference because that 23 error rate, that yes/no error rate is going to be extrapolated 24 to the population, and it also affects how big a sample size you 25 pick and what assurance levels you can give. So it's important 1411 1 from that standpoint. 2 It's also important because not only is the dollar 3 difference used in this notion of yes/no, which is the attribute 4 sampling, it also becomes a variable on the other side, which 5 it's called variable sampling, where you sum up those dollar 6 amounts. So there's really -- going all the way back to the 7 2003 Plan, it was always clear to me from the 2003 Plan and from 8 the 2007 Plan that the fundamental design was based on attribute 9 sampling, it's a yes/no question. 10 And back in the 2003 trial there was some discussion that 11 they may eventually do a variable type sample. They may gather 12 that information. In the course of this trial it's become clear 13 that they have done that activity. In Ms. Hinkins' rebuttal 14 report, she's made it very clear, and in the administrative 15 record you can see they have gathered that additional 16 information on how much the dollar difference is. 17 Well, really, the attribute sample piece, the yes and no, 18 there's no need to set some arbitrary level for yes or no when 19 you're going to collect that information on the dollar side. So 20 it's important here, if there is any difference, it should make 21 a difference in the attribute side, whatever the size of that 22 is, because that has ramifications on the rest of the sampling 23 exercise. 24 Q. And does it make a particular difference, since the 25 statements that are being made are based on aggregate amounts as 1412 1 opposed to statements particularly to an individual Indian trust 2 beneficiary's account? 3 A. Well, here the statements that are made in the aggregate 4 are really driven off of both the yes/no question and off of the 5 dollar size question, so it makes a difference in both 6 instances. 7 Q. And therefore whether it's a dollar or 58 cents, it may be 8 significant when you're extrapolating to the entire population, 9 correct? 10 A. They need to be tracked down. As Dr. Scheuren correctly 11 pointed out, a mistake of 58 cents, you need -- if it's not 12 reconciled, you need to spend time tracking it down, 13 particularly when you're sampling, because if that transaction 14 should have been $58 or $58,000, that's going to make a big 15 difference as to how you extrapolate the implications up to the 16 population that you're able to extrapolate to. 17 Q. We've been discussing on this page the sample debit 18 transactions. Does the same concern exist with regard to 19 sampled credit transactions? 20 A. It does, but there's an additional layer to the flaw here 21 in thinking. And that is, we read here there are 11 22 underpayment differences totaling 341, and 25 overpayment 23 differences totaling 853. And then the conclusion is the net 24 overpayment was 512. 25 And this is another difference that Dr. Hinkins and I have. 1413 1 And this goes directly to my notion of sample design. It's 2 irrelevant, the last part of that sentence is irrelevant to an 3 individual account holder. It makes perfect sense in estimating 4 exposure from -- liability in a litigation setting, but if I'm 5 the individual that was underpaid $341, just take this example, 6 if someone else was overpaid $853, that's irrelevant to me. You 7 can't offset those transactions. 8 And the sample design and what was reported in the sampling 9 really gets this notion of the net. That's the way it's 10 reported by the accountants, that's the way it's reported in the 11 2007 Plan, and that netting is inappropriate in the context of 12 trying to answer the question for individual account holders. 13 Q. Is it your understanding, then, that at the completion of 14 the statistical sampling exercise, that the statement that would 15 be made to the account holder who is underpaid $341 is that his 16 stated balance is more likely than not accurate, and he would 17 not be told he was underpaid $341? 18 A. It would have to bear itself out, how the HSAs were 19 developed. But from what I've seen so far, it would be that 20 your recorded transactions are probably right. They might be 21 off by 1 percent, 4 percent, depending on credits or debits. 22 But the individual that was underpaid the 341, from what I've 23 seen so far, the only answer would be it's probably right within 24 about 1 percent. 25 Q. Can this concern that you've discussed be cured with a 1414 1 larger sample size? 2 A. No. All it would do is it would make the estimate of the 3 underpayment or the overpayment, and if they go the next step of 4 netting it, it would make each one of those a little more 5 precise, but it's not going to solve the underlying problem of 6 the inability to say something about the individual account 7 balance. 8 Q. And would that information enable the defendants to confirm 9 that individual transactions were accurate? 10 A. I'm not sure I follow that question. 11 Q. Okay. If you have a mean overpayment and mean underpayment 12 associated with a posted transaction on two electronic systems 13 and you expanded the sample size, you could not confirm based on 14 that information that the individual transaction was accurate, 15 could you? 16 A. You wouldn't have any basis. 17 Q. It provides nothing for you, does it? 18 A. Correct. 19 Q. I'd like to move on to problem No. 4, which you've 20 described as inappropriate sampling unit. Can you please 21 explain what you mean by an inappropriate sampling unit? 22 A. The issue that comes to bear here is we're trying to 23 understand what happened in accounts. Inside of the accounts 24 there are transactions. 25 I think that DOI and certainly NORC recognized early on 1415 1 that sampling accounts would be problematic, because there's a 2 likelihood, a reasonable likelihood that there would be an error 3 in one of the transactions in the account. So as a matter of 4 fact, early on in this process, when there were discussions 5 about sampling accounts, sampling transactions, and they were 6 weighing out the strengths and weaknesses, it was specifically 7 articulated that even if you have very, very low error rates in 8 the transactions, which is really contrary to what you would 9 assume given all of the other reports from accountants and GAO, 10 but if you assume it was going to be a very low transaction 11 error rate, by the time you lumped all of those transactions 12 into an account, what you're going to end up with was a high 13 likelihood that you're going on have the account balance 14 misstated. 15 So looking at the transactions, let's say you assume 16 there's only 1 percent error rate. Given the average size of 17 these accounts, by the time you look at each account and say 18 okay, there's a 1 percent chance that transaction's wrong, 1 19 percent chance that transaction's wrong, by the time you get to 20 the end of -- all the transactions in that account, even 21 assuming a 1 percent error rate, you would say you can be 75 22 percent sure that account balance is misstated. 23 If the error rate goes up at all, to 2 percent or 3 24 percent, effectively you're a hundred percent sure that every 25 account balance is misstated. So it's the wrong unit to be 1416 1 looking at. You shouldn't be looking at transactions. Really 2 what should be done here, if you were going to go through this 3 sampling exercise, is it should be done on an account basis. 4 But the lack of information really tied their hands and they 5 weren't able to do that. 6 Q. Let's take a look at an exhibit from the administrative 7 record, and it's identified as Bates No. 11501. Do you recall 8 reviewing this document? 9 A. Yes, I do. 10 Q. As you can see, this is a November 24, 2002, document 11 that's identified as draft from the Office of Historical Trust 12 Accounting, and the headnote is "Statistical sampling strategy," 13 and underneath that is "the scientific logic of using sampling 14 approaches." Is that correct? 15 A. Yes. 16 Q. I'd like you to turn to what is Bates page No. 4 of this 17 document, and I'd like you to focus in on the second paragraph. 18 It's highlighted. What you've been discussing with regard to 19 the concern about error rate in an account versus error rate in 20 the transaction, is the highlighted section one of the concerns 21 that you've had? 22 A. Yes. This articulates exactly what I was talking about. 23 It assumes a much, much -- a tenth of the error rate I was just 24 talking about. Here assuming that 0.1 percent of the 25 transactions are wrong, the account error rate would explode 1417 1 upwards of 30 percent. The example I gave was if you assume it 2 was 1 percent error rate, it would balloon up to 75. So this is 3 exactly the issue that I was talking about, and it was 4 identified fairly early in the sampling exercise, as we see 5 here, by the Office of Trust Accounting document. 6 Q. So is it fair to say that if the goal was to substantiate a 7 near zero error rate, sampling accounts would not allow you to 8 reach that goal? 9 A. You could establish a very small error rate for the 10 transactions, potentially, but it would not result in a high 11 likelihood that the account balances are correct. 12 Q. Now, is the determination of an error rate part of 13 Defendants' accounting efforts? 14 A. It is. And we keep coming back to this notion because it 15 underpins everything. What is the error rate? And that is 16 defined by what is the error. So it's very important to 17 understand what that error rate is and what comprises that error 18 rate as part of conducting the sampling. 19 Q. In that regard, I'd like you to turn your attention to an 20 exhibit, another exhibit from the administrative record. It's 21 Bates numbers 57-30-1. Have you reviewed this exhibit before? 22 A. Yes, I have. 23 Q. This is entitled "Managing historical accounting records." 24 It is undated. I'd like you to turn your attention to Bates 25 No. -- page No. 3. 1418 1 THE COURT: It's also unidentified, isn't it? 2 MR. GINGOLD: Yes, it is. But it was in the 3 administrative record, Your Honor. It's identified with an HTA 4 Bates stamp number. We assume it's something that was either 5 considered or relied on in making a decision because it was part 6 of the record. 7 THE COURT: All right. Go ahead. 8 BY MR. GINGOLD: 9 Q. In the middle of the page on Bates No. 3, we have 10 highlighted a section. And you've discussed the problem of 11 missing records, correct? 12 A. Yes. 13 Q. And you've discussed the errors associated with that. 14 A. I have. 15 Q. The highlighted section of this document specifically 16 states, "This warns us that we need to have a strategy that 17 resolves the missing record problem for disbursements in a way 18 that does not treat the failure to find the supporting 19 documentation as an error in the account." 20 Now, what is your understanding of that statement? 21 A. Well, this goes to the notion of the sample size 22 determination and what really the exercise is trying to 23 accomplish. At this point in time, it's referencing this 1972 24 to '92 tribal reconciliation. This was the reconciliation I 25 understand that was performed by Arthur Andersen. 1419 1 I've reviewed that tribal reconciliation. And the tribal 2 reconciliation found that even after making a whole series of 3 amendments to the standard by which they were, quote, 4 reconciling transactions, I believe they started out at GAAP and 5 GAAS standards, and after a series of nearly a hundred 6 modifications to their work order, at the end of the day they 7 arrived with what the BIA instructed them was the standard to 8 which it should be reconciled. 9 Even after lowering that standard all the way down, they 10 still had a substantial proportion of the documents that they 11 couldn't reconcile, that had missing supporting documentation. 12 If that was the case, and it was something in the 12 or 13 13 percent neighborhood that they could not reconcile, that would 14 be a huge implication on the sampling exercise, because if you 15 assume a 13 percent error rate, if you treat all of those 16 unreconciled as errors, then the sample size grows -- well, not 17 even proportionally, it would grow even faster than that. 18 There's a large increase in the required sample size, and 19 there's potentially a decrease in the assurance rates that would 20 be able to be provided. 21 So I think it's correctly identified here that if they're 22 going to treat these missing documents and unsupported documents 23 as errors, they're going to have -- it would have to be a 24 substantially different sampling exercise than what was 25 conducted. 1420 1 Q. Is it fair to say they wouldn't be able to achieve a near 2 zero error rate? 3 A. Correct. If you don't find a way to ignore those as 4 errors, you're not going to get close to a zero error rate; your 5 expectation going in would be something in the 12, 13 percent 6 area. 7 Q. Is it your understanding that this particular concern on 8 how to treat missing documents vis-a-vis error rate is something 9 that was actually underscored by the Defendants in the 10 administrative record? 11 A. Yes, it was. 12 THE COURT: If there's no objection to that question, 13 I guess I'll let it stand, but this is the same document you 14 showed to Dr. Scheuren and he couldn't identify it. So I'm not 15 taking very much from this. Go ahead. 16 BY MR. GINGOLD: 17 Q. Let's go to Bates page 10 of the same document. And if we 18 can turn to the second paragraph, which is the first highlighted 19 paragraph on this page, you will note that that same warning 20 with regard to treating missing documents is restated, isn't it? 21 A. Yes, it is. 22 Q. And you have reviewed this entire document, have you not, 23 Mr. Duncan? 24 A. I have. 25 Q. Now, one of the things that you've pointed out is documents 1421 1 are resolved depending on coded numbers, correct, 1, 2, 3, and 2 4 -- or reconciliations are resolved -- 3 A. Correct. There were four codes for the reconciliation 4 efforts. 5 Q. All right. And what is your recollection of code No. 1? 6 A. It changes modestly over time, but the consistent 7 definition of an accounting code 1 is it has a directly 8 supporting document, it has been recomputed using known 9 Department of the Interior methods, or it has been substantiated 10 with third-party documentation. 11 Q. And what about a code No. 2? 12 A. Code No. 2 is this term, alternative procedure. I've 13 really, despite some extended efforts to try and find something 14 as to what that means, I've not seen anything other than it's 15 called an alternative procedure. 16 Q. As a matter of fact, you've reviewed portions of the 17 administrative record, isn't it true? 18 A. The large majority of the administrative record I've 19 reviewed personally. 20 Q. Did you find anything in your review of the administrative 21 record which explained the alternative procedures in detail? 22 A. Not in any detail that I could decipher what exactly was 23 the guidance to the individual accountant that's doing the 24 reconciliation. 25 Q. If we can go to the middle of this page, you'll see a 1422 1 highlighted A, reconciled with directly relevant documents 2 without discrepancy, and you'll see A, B, C, D, E, 3 unsubstantiated. Is A consistent with your understanding of a 4 1-rated reconciliation? 5 A. Well, accounting code 1 would be both A and B. There's 6 sort of two steps to it. The first is the reconciliation. Can 7 we tie it back to some document that substantiates it's either 8 reconciled yes or no, and then the second question is is there a 9 discrepancy in that reconciliation. Did we find a supporting 10 document that's different or did we find supporting documents 11 that support it. So accounting code 1 deals with A and B. 12 Q. So accounting code 1 even includes documentation which may 13 be in conflict with the posted entry, correct? 14 A. From the standpoint of it being reconciled, yes or no, it 15 is. 16 Q. And if we look at C and D on this page, "inferred 17 reconciliation, strongly persuasive," and "inferred 18 reconciliation, persuasive to some extent," do you have an 19 understanding of what C and D mean? 20 A. Those map in some degree into accounting code 2 and 21 possibly 1. 22 Q. But again, you don't have -- you haven't been able to 23 review any detail which explains the accounting codes, have you? 24 A. Other than the ASM, but that does not tell me definitively, 25 and the information about the results of the sampling don't tell 1423 1 you even whether or not, for accounting code 1, were they 2 reconciled with a directly relevant document, or were they an 3 inferred reconciliation. You can't tell in that bucket that's 4 grouped as reconciled code 1 which of those happened. 5 Q. And in code 2 sound judgments are permitted, correct? 6 A. I believe code 1 and code 2, judgment is permitted. 7 Q. On the paragraph underneath the A, B, C, D, E descriptions, 8 there seems to be an explanation of A through E. Did you review 9 that before? 10 A. I did. 11 Q. What is your understanding of that explanation? 12 MR. WARSHAWSKY: Objection. This document speaks for 13 itself. 14 THE COURT: Thank you. I'm going to sustain the 15 objection. I tried to -- I mean, this is just a document that's 16 found in the administrative record. Nobody quite knows what it 17 is. I'm more interested in this witness's opinion than what he 18 thinks of some other paper that's in the administrative record. 19 BY MR. GINGOLD: 20 Q. Do you believe reconciliation should be performed without 21 direct source documents? 22 MR. WARSHAWSKY: Objection. No foundation for the 23 witness -- 24 THE COURT: I'll allow that question. I'll allow 25 that. 1424 1 THE WITNESS: Ultimately that's what underpins all of 2 the sampling exercise, is what is reconciled and what is not. 3 And as a statistician, if it were tasked to me to try and 4 perform this particular sampling exercise, it makes a huge 5 difference whether or not something is reconciled with a 6 directly supporting document, or whether or not it has been used 7 through some judgment or recalculated or something like that. 8 So I would not be comfortable preparing opinions or doing 9 an extrapolation where you don't provide to the consumer of this 10 information what the difference was, which ones were reconciled 11 where there was actually back to a source document that says, 12 and it matched. Which ones were recomputed? Because quite 13 frankly, if I was looking at one that was recomputed, I'm not 14 really sure that you would expect to find any error. Or one 15 that is an alternative procedure. I don't know what an 16 alternative procedure is, but I know what it's not. It's not 17 linked back to a source document, directly tied to that source 18 document. It's not a third party document. It's not inferred 19 from the recomputation of the efforts. 20 So everything in accounting code 2, what that means is it's 21 something other than that, but I wouldn't really have any idea 22 why you would expect to find an error under an alternative 23 procedure. 24 So when all of those get lumped together for the 25 calculation of the error rate, in my mind that makes anything 1425 1 based on that error rate meaningless, because you just don't 2 know how you got to this notion of there being a 1 percent error 3 rate. 4 BY MR. GINGOLD: 5 Q. And specifically, is what you're saying that you as a 6 reviewer of the information would not be able to make a decision 7 as to what is an error in a particular transaction? 8 A. Correct. The entire sampling is based on this notion of 9 the error rate, and if you don't have any understanding of what 10 constituted the error, or have any confidence that an error has 11 a meaning other than it could be any of these other procedures 12 and not be an error, it would be nearly impossible to make any 13 informed decision on that basis. 14 Q. I'd like to bring to your attention another document from 15 the administrative record. It is identified as Bates No. 16 60-12-1. This is a September 22, 2003, document entitled 17 "Accountant's report on the reconciliation of the Eastern Region 18 land-based non-interest individual Indian money transactions." 19 Have you reviewed this document before, Mr. Duncan? 20 A. Yes, I have. 21 Q. I'd like you to turn your attention to Bates page 4, and 22 I'd like you to look at this through page 10. And as you can 23 see on Bates 4, you have categories which are identified as 24 trans ID, transaction amount, throughput transaction amount, 25 reconciled transaction amount, total variance, and accounting 1426 1 code. Do you have an understanding of what this is? 2 A. I do, and this document was very important in creating and 3 drafting our opinions. This is the only information that was 4 provided as part of the administrative record where we actually 5 see what the accountants did in the process of doing the 6 reconciliation. 7 So what we have here is for the Eastern Region, this 8 document goes down for the 289 transactions that were sampled in 9 the Eastern Region, and what we see going across the line is 10 there's an identifier, the 001, that just identifies the 11 transaction, we can see that in the electronic ledger, this was 12 a $636.82 transaction. This was reconciled, if you go all the 13 way to the right column, you can see this is an accounting code 14 1. So that tells us that this was either directly reconciled, 15 recomputed, or subject to third party documentation. It was one 16 of those three things happened. And that reconciled to an 17 amount of $636.82 with no variance. 18 So for this particular transaction, it's possible, although 19 we can't tell, it's possible that this had an actual transaction 20 that would show us exactly this is the right amount. It was a 21 lease collection or check disbursement, depending on positive or 22 negative, but there's this notion that you can see this is 23 exactly how much came in. That's a possibility. It's also a 24 possibility that it was just recomputed using some known 25 Treasury method or Department of Interior method. 1427 1 Now, importantly, though -- maybe if I can just have you 2 pan down. 3 Q. Drop to let's say trans ID 48. 4 A. Good. That's one example. So we can actually see here 5 transaction 48, it's in the electronic ledger at $162.23. It's 6 also in accounting code 1. So what happened here is there must 7 have -- however it worked in accounting code 1 -- we don't have 8 the work papers. I can't see whether or not there is a check or 9 there is a lease income. All we know, we have this piece of 10 information. But it tells us that somehow they established that 11 this particular account was off -- it's only off by a penny, but 12 this transaction was off by one cent. 13 Now, that, Your Honor, perhaps to put some context to what 14 we were talking about before, but in answering a question in an 15 attribute sample, my position would be that's an error, that's a 16 mistake. It's only a penny, but that penny would be caught in 17 the second part of this, which is the variable calculation. We 18 can gather that up to quantify the magnitude, but it was 19 recorded incorrectly. 20 For purposes of the Eastern Region, though, that 21 transaction was concluded that there was no error, because it 22 fell below the de minimis rule. So this, I think, certainly 23 helped us understand when we were looking at this, exactly what 24 was happening at the individual reconciliation level. 25 So in the Eastern Region, you'll recall, Your Honor, it was 1428 1 NORC's conclusion that there were no errors. There's a series 2 of I believe 12 or 13 -- maybe 13 of these same kinds of 3 mistakes that are off by very small amounts, but at the end of 4 the day, NORC concluded there are no errors in this 5 reconciliation at all. 6 Q. And let's turn to Bates No. 5. And if we could focus in on 7 the top half of this so Mr. Duncan can read this. 8 You see there's additional variances of a penny and two 9 cents on this particular page, correct? 10 A. Yes. 11 Q. And again, you're unable to determine how that was 12 determined because you don't have the work papers, correct? 13 A. That's correct. We do know that these are accounting code 14 1. That tells us something about how it was reconciled, but 15 nothing specific. 16 Q. Let's drop to the bottom half of the page. Were there 17 additional variances identified, each of which is only a penny, 18 correct? 19 A. Correct. 20 Q. I'd like to turn to page 6. If you see on page 6 there are 21 additional transactions which are identified as a penny, 22 correct? 23 A. Correct. I believe on this page there's also some 24 negatives. There's positives and negatives. 25 Q. Does it make a difference if it's a positive or negative 1429 1 variance? 2 A. Well, this goes to the notion of the netting. At the end 3 of this report, you'll see that the net variance is provided, 4 and it does. It nets out the positives and negatives between 5 these transactions. 6 Q. What I'd like you to turn your attention to now is Bates 7 No. 9. And let's just focus on the top half of this first so 8 it's more easily readable. You note that these reconciliations 9 are done in accordance with accounting code 2? 10 A. Correct. There are I believe 42 that were accounting code 11 2 in this sample. 12 Q. But there's no variance? 13 A. I wouldn't expect there -- I don't know what the 14 alternative procedures are, but again, I know what they're not, 15 so I wouldn't expect there to be a variance here under 16 accounting code 2. 17 Q. And why wouldn't you expect a variance in accounting code 18 2? 19 A. Well, it's not linked to a directly supporting document, 20 it's not recomputed, whatever the recomputation means, and it's 21 not a third-party source; it's some alternative procedure. So 22 my guess is it's going to be highly unlikely under this 23 alternative procedure, whatever those methods ultimately turn 24 out to be, it's going to be highly unlikely that you're going to 25 be able to contradict the dollar amount with the alternative 1430 1 procedures; ergo I wouldn't expect to see an error. 2 Q. I'd like you to turn to Bates 10. And if you look at the 3 reconciliation columns above, what does that tell you? 4 A. Well, these are all accounting code 4, and I'm sure, 5 Your Honor, you've seen -- the definitions are actually provided 6 on the bottom. Transaction 4 is there's no evidence that can be 7 found, no alternative analyses, no alternative procedures. 8 These are simply ones that there was no way to reconcile. 9 Now, there's a little bit of a twist in the Eastern Region 10 because it was done early as part of the LSA, and the 11 transactions, the $105 to the $200, all of those transactions 12 are more than a hundred dollars. Here the certainty stratum 13 that was utilized was not a hundred thousand dollars and 14 greater, but it was a hundred dollars and greater. So there was 15 an attempt to reconcile all of these as well. 16 Ultimately it turns out these six transactions were not 17 able to be reconciled under anything, so they were lumped into 18 accounting code 4. And as you can see, there's no variance 19 associated with it, again, because you can't reconcile it, there 20 would be no expectation of being able to calculate a variance. 21 Q. And is it your understanding there's a zero error rate as 22 determined by NORC in the Eastern Region? 23 A. That's correct. 24 Q. Now, are you aware, other than -- you've discussed the fact 25 you haven't seen work papers which help explain how the 1431 1 accounting codes have been employed, correct? 2 A. They would be very helpful in understanding what 3 constitutes these different codes. 4 Q. Is there any other evidence in the administrative record 5 that you've reviewed that helps you understand what the error 6 rates are? 7 A. Well, there's the inclusion at the beginning of the Eastern 8 Region, there's a discussion as to what the error rates -- you 9 know, that there were no error rates -- there was a no error 10 rate conclusion with respect to this particular project with the 11 Eastern Region. 12 Q. And let's turn on this exhibit to Bates page No. 2. Is 13 this what you're referencing on the bottom of the page? 14 A. Yeah. This conclusion, I think it rolls to the next page, 15 but this was the basis for what was then provided in some of the 16 summary information about the national sample, as it related to 17 the Eastern Region. 18 So there were a total of 289 transactions. They said they 19 directly reconciled 241 of those, or they were able to directly 20 reconcile it by using directly relevant documents. Again, what 21 that means is, those were all accounting code 1, and it tells 22 you what the three different definitions of accounting code 1 23 are here that I've articulated for the Court. 24 Q. And then it goes on on the bottom of the page, carrying 25 over to Bates 3, dealing with 42 transactions which were 1432 1 reconciled, correct? 2 A. Correct. 3 Q. Using alternative procedures, correct? 4 A. Correct. It tells me nothing more than it was in fact an 5 alternative procedure. And this again links to what we saw at 6 the bottom of that last table, which is the definition of the 7 accounting codes. 8 Q. And finally, at the end of this explanation by Deloitte & 9 Touche it talks about six transactions which Deloitte was unable 10 to reconcile, correct? 11 A. That's correct. 12 Q. What does this tell you? 13 A. Well, again, the error rate, if I were designing this as an 14 attribute sample and I was going to collect the information from 15 the variable sample anyway, then I would think that any error 16 should be counted as an error, because it's going to be 17 extrapolated up to a large population in the electronic ledger 18 era. And that information indicates, based on the number of 19 errors we saw there, it's going to be something in the 6-1/2 20 percent range, would be the errors in that particular instance. 21 But by defining an error as having to reach a certain 22 level, you're able to reach -- DOI is able to reach a lower 23 conclusion about the error rate, therefore draw a much smaller 24 sample size. 25 Q. And what is your understanding of what NORC actually stated 1433 1 with regard to the Eastern Region? 2 A. My understanding is that they concluded that they 3 reconciled, of the 289, they're 98 plus percent certain that 4 there's a very low error rate based on the fact that they found 5 no errors. 6 Q. Let's look at document 51-4 at 4. You've reviewed this 7 document, haven't you, Mr. Duncan? 8 A. Yes, I have. 9 Q. And this is a statistical evaluation of preliminary Eastern 10 Region sample results, March 2004, by NORC, correct? 11 A. Correct. This is a summary of the study that we just 12 looked at by Deloitte & Touche. 13 Q. Let's go to Bates 4. Let's focus in on the highlighted 14 portion of this page. What does this tell you, Mr. Duncan? 15 A. Well, this is what we talked about this morning, is this 16 notion that because of the way the error rate was defined, that 17 NORC can say that no errors were found. And on that basis they 18 can make the kinds of statements that you see ahead of that, at 19 the beginning of that sentence. 20 Q. And so no errors were found, basically, notwithstanding the 21 issues that you've just discussed for the last several minutes, 22 correct? 23 A. Correct. 24 Q. Now, this is only the Eastern Region, though. Why is this 25 important? 1434 1 A. Again, there's nothing else in the administrative record 2 that allows us to do this level of detail, to actually look 3 behind the curtain. This is the only part of that study that 4 allowed us to do so. 5 Q. And does it also discuss the approach that NORC has been 6 taking with regard to how it is reconciling posted transactions, 7 or not? 8 A. Well, NORC's not doing any of the reconciliation work. 9 That's all done by the accounting firms. NORC designs the 10 sample, provides the sample list, and then the accounting firms 11 go through the process of doing the reconciliation work. 12 Q. What about the plan itself, the historical accounting plan? 13 Let's go to Bates 33-2-14. Now, this itself, if you look at the 14 middle of the page, this deals with tribal reconciliation, 15 correct? 16 A. Correct. 17 Q. And you're talking about error rates generally with regard 18 to NORC's work, correct? 19 A. Correct. 20 Q. And tribal reconciliation work was used as a basis for 21 taking the approach that was done in the 2007 Plan, correct? 22 A. At least in part, yes. 23 Q. Now, you also discussed the fact earlier that the tribal 24 reconciliation report had an error rate that was significantly 25 greater than 1 percent, didn't you? 1435 1 A. Well, the tribal reconciliation work had an unreconcilable 2 rate in the area of about 13 percent, and this is being 3 pulled -- I believe this comes out of the 2007 Plan directly. 4 And this goes to the notion of saying, well, doing the 5 historical accounting, we need to make an assumption. Before we 6 start, we need to make an assumption about what's the error rate 7 going to be. 8 And when they looked at things, like they said here we 9 looked at previous work like the tribal reconciliation project, 10 if you looked at that tribal reconciliation project and your 11 thinking was if a transaction is unreconciled, I will treat it 12 as an error, then the assumption you would make about the 13 underlying error rate is it would be something in the 13 percent 14 area. 15 But in fact, that's not the assumption that they made. The 16 assumption that is made, and it's all throughout the 17 administrative record, the assumption that was made was it would 18 be a very low error rate. And in order to make that assumption 19 based on what they cited in the plan, you must have made the 20 assumption that an unreconciled transaction is not an error. In 21 other words, if there's no supporting document, it's okay. 22 That's the only way that you could cite that study as a premise 23 for a very low error rate. You have to assume that unreconciled 24 is not an error. 25 Q. Do you believe it's reasonable to assume a 1 percent error 1436 1 rate based on the information in the tribal trust reconciliation 2 report? 3 A. Base on the tribal reconciliation report, based on the 4 paragraph 19 work, based on the Rosenbaum work, based on 5 everything that I have observed, I don't think it's reasonable 6 to start -- if you're going to treat unreconciled as an error, 7 it's not a reasonable basis to conclude that it's going to be a 8 very low error rate. There's just too much missing information. 9 Q. So how meaningful is the statement with regard to the 10 probability that transactions are accurate and balances are 11 correct if in fact the assumption is unreasonable? 12 A. Well, if it's premised on the notion that in order to 13 achieve an error it has to be not all the things that we talked 14 about, it can't be accounting code 1, it can't be accounting 15 code 2, in order for it to be an error it has to reach this 16 notion of there being some contradictory underlying document, it 17 really makes any of the implications to try and say something 18 about the accuracy and completeness of the account transaction 19 listing and the account balances, it really makes anything drawn 20 from that meaningless. 21 Q. You were also asked to rebut, Mr. Duncan, the expert 22 reports of Drs. Lasater, Scheuren and Hinkins; is that correct? 23 A. That's correct. 24 Q. Let's start with Dr. Lasater. Based on your recollection, 25 what were Dr. Lasater's opinions that he expressed to this 1437 1 Court? 2 A. Dr. Lasater's opinions really focused on the expectation 3 for how the paper ledger era would be executed. 4 Q. Just the paper ledger era? 5 A. As it was expressed to the Court. In his written opinion 6 he had some statements that were very similar to what 7 Dr. Scheuren had been saying. So as I read it, the Judge only 8 wanted to hear on the part that was not duplicative. 9 Q. If Dr. Lasater stated that the proposed methodology for 10 testing transactions in the paper ledger era is reasonable, 11 would you agree with that? 12 A. Absolutely not. 13 Q. Why not? 14 A. Well, first of all, I don't know what basis Dr. Lasater has 15 for reaching that conclusion. I've reviewed the administrative 16 record, I've reviewed the NORC reports. There is precious 17 little information about what exactly is going to be done in the 18 paper ledger era. So I don't think there's an adequate basis to 19 reach the conclusion that it's reasonable. We don't know what 20 it is yet. 21 Q. Did Dr. Lasater provide the basis for his opinion in that 22 regard? 23 A. Dr. Lasater indicated that he had had some discussions with 24 NORC about how they were going to do that, but nothing really 25 materially beyond what we knew from the administrative record. 1438 1 Q. Did he reference any documents that supported that opinion? 2 A. No. 3 Q. If Dr. Lasater stated in his opinion to this Court that the 4 proposed difference testing to evaluate statistically whether 5 the paper ledger and electronic era transactions were processed 6 by the same quality of bookkeeping is sound, would you agree 7 with that opinion? 8 A. I definitely would not agree with that, on several levels. 9 The notion -- the hypothesis that Dr. Lasater's putting forth 10 and that is vaguely described in the plan is that whatever was 11 done over here in the electronic ledger era, what we're going to 12 do is just go look in the paper ledger era, and if in fact 13 there's no material difference between what we see in the paper 14 ledger era versus the work that was done in the electronic 15 ledger era, then we'll assume that those two populations are the 16 same. 17 Well, every bit of information I've reviewed in the 18 administrative record indicates that that is not the case. 19 There's not even a reasonable basis to start with that 20 hypothesis. 21 The only way that that hypothesis would potentially make 22 sense is if the criteria upon which you compare the electronic 23 ledger era to the paper ledger era is the same criteria, with 24 this very, very high hurdle to create an error. Something where 25 you would have almost no expectation of finding the error. You 1439 1 might be able to reach the end conclusion that they're the same, 2 but it's only because that which you are seeking to detect would 3 not be detectable by the systems that you're using. 4 Q. Does the FTI September 30, 2007, report provide the support 5 for Dr. Lasater? 6 A. No. 7 Q. And why not? 8 A. There's just not enough information in that report to 9 determine that that's definitively the way that it's going to be 10 done. 11 Q. I'd like you to turn your attention to another exhibit from 12 the administrative record. It is identified as 55-17-19. Have 13 you reviewed this report? 14 A. Yes, I have. 15 Q. I'd like you to turn to page 37 of this report. And I'd 16 like you to focus on the second bullet point, which is 17 highlighted on this page. It states: "A review of the 18 paragraph 19 search indicates that records recovery diminished 19 from the 1930 to 1950 time frame and fell off before 1930." 20 What is your understanding of that statement? 21 A. Well, this is just one piece of information. When I said I 22 reviewed the administrative record and it was clear to me that 23 the expectation should be that the records in the paper ledger 24 era are significantly worse than the electronic ledger era, this 25 is simply something -- I believe this was presented by 1440 1 Mr. Edwards. This is simply documenting that fact, that the 2 records get worse the further you go back in time, which is not 3 a particularly controversial statement. 4 Q. So when Dr. Lasater made or offered his opinion, he didn't 5 address the concern of the diminishing availability of records, 6 did he, as you go further back in time? 7 A. He did not. 8 Q. Did Dr. Scheuren? 9 A. I don't believe that he made any specific comments or 10 opinions about how they were going to deal with this particular 11 issue. 12 Q. So if the observations were made by Dr. Scheuren and 13 Dr. Lasater that the two eras, as they define paper and 14 electronic, are substantially similar, you see no basis to 15 support those opinions, do you? 16 A. I think the hypothesis itself is a waste of time to even 17 bother to test. Everything would be contraindicated here. The 18 hypothesis should be that they're fundamentally different based 19 on everything that we've seen. 20 Q. And when you say everything, do you mean audit reports? 21 A. Audit reports, reconciliation reports, GAO reports, I mean 22 reports to Congress, everything I've seen indicates that the 23 records drop off substantially as you move back in time. And 24 again, it's because there's missing documents. There's records 25 that have been destroyed. 1441 1 Q. Do you recall any discussion by Ms. Herman about how the 2 record and bookkeeping systems among the various regions and 3 agencies were different over time? 4 A. Yes. She testified that they varied over time and between 5 agencies. 6 Q. So it would be very difficult in your opinion to reach the 7 conclusion that the paper records era and the electronic records 8 era are sufficiently similar to draw inferences from the 9 electronic era to the paper era, correct? 10 A. I think that's a fair statement. 11 Q. Now, Dr. Scheuren indicated that -- didn't Dr. Scheuren 12 indicate that no additional analysis or testing is necessary in 13 that regard? 14 A. Dr. Scheuren indicated that there was no additional 15 analysis done in the electronic era. 16 Q. Did he indicate whether or not additional analysis is 17 necessary? 18 A. In the electronic era? 19 Q. Correct. 20 A. He indicated that it was not. 21 Q. Now, did Dr. Scheuren take into consideration that the LSA 22 project was designed to estimate risk exposure, not transaction 23 accuracy? 24 A. No. Again, the notion of the application of the faulty 25 sample design was not addressed as a rebuttal to my opinion by 1442 1 Dr. Scheuren. 2 Q. And did either Dr. Lasater or Dr. Scheuren take into 3 consideration in their opinion the fact that the target 4 population is different from the sample population due to 5 missing and omitted data? 6 A. I don't believe that either one them specifically addressed 7 that. It was really addressed by Dr. Hinkins. 8 Q. Do you know whether or not anyone has been tasked with 9 establishing the accuracy of account balances? 10 A. I don't know if anyone -- I don't know of anyone that has 11 been. I do know that NORC specifically stated they were not 12 tasked with designing a sample that would address that issue. 13 Q. And that was in Dr. Hinkins' rebuttal report, wasn't it? 14 A. That's correct. 15 Q. What effect, if any, does that have on your view of the 16 adequacy of the statistical sampling plan? 17 A. Well, I think fundamentally Dr. Hinkins and I agree on 18 several topics. This is one of them. My criticism, my concern 19 during my time in 2003, and at my preparation for the trial here 20 in 2007, has been that you can't take this kind of transaction 21 sampling and apply it to accounts. And the rebuttal point on 22 that issue from Dr. Hinkins was that NORC had never been asked 23 to do a sample design that would address accounts. 24 So I think fundamentally we're in agreement on that point. 25 What has been done in the LSA sampling will not address 1443 1 accounts, account balances. 2 Q. But if the LSA sampling was designed for purposes of 3 limiting liability or error, it could be used as a sound basis, 4 could it not? 5 A. If it's limited to the electronic ledger era, and limited 6 to the documents that were available for sampling, then yes. 7 Q. And similarly, couldn't NORC's LSA results for purposes of 8 the litigation support accounting be considered as having an 9 appropriate measure of error? 10 A. I'm sorry, I couldn't hear you over the coughing. 11 Q. With regard to NORC's LSA results, the litigation support 12 accounting results, and its use of error in that regard, would 13 it be inappropriate use of error in the LSA that it is using 14 versus what we're talking about, the establishment of accurate 15 account balances? 16 A. No. Again, I think the error rate is too narrowly defined 17 to be useful even in that context. 18 Q. And does a qualitative meta-analysis report assist 19 Dr. Scheuren in his opinions with regard to the appropriateness 20 of drawing inferences from the electronic record era to the 21 paper record era? 22 A. I don't believe it provides any reasonable basis for making 23 that conclusion. 24 Q. Why is that? 25 A. Well, the meta-analysis, first of all, we obtained some 1444 1 information from the meta-analysis. There were many of the 2 things that had been referenced in the meta-analysis that were 3 not provided to us. Of the documents that were provided to us, 4 many of them had missing pages, which from our review appeared 5 to be pages that were material, that would have potentially 6 influenced the way that that meta-analysis was reviewed. 7 But in any event, the documents that we were provided and 8 did look at, many of them were internally inconsistent with the 9 conclusions that had been reached in the meta-analysis. So I 10 don't feel that the meta-analysis forms a reasonable foundation 11 or basis upon which to make a conclusion that the paper ledger 12 era is the same from a bookkeeping standpoint as the electronic 13 ledger era. 14 Q. Now, I'd like you to look at the fourth demonstrative you 15 have on the original exhibit, where you actually identified as 16 you understood it the NORC solution in redefining the target 17 population. Do you recall preparing this demonstrative? 18 A. I do. 19 Q. Could you please explain it? 20 A. Well, this is another area where I think Dr. Hinkins and I 21 fundamentally are in agreement. I identified this issue prior 22 to Dr. Hinkins issuing her report, that the target population, 23 that there were things missing and omitted from the target 24 population. And Dr. Hinkins in her rebuttal clarified that in 25 fact that was not what they were calling the target population. 1445 1 They, NORC, had said the target population is actually the 2 population from which we drew the sample. 3 And if that's the way you define the target population, 4 then my criticism falls away. You can actually draw an 5 inference from the population from which you've sampled. So to 6 the extent that NORC is now clearly delineating that the target 7 population is actually only that population from which the 8 sample was drawn, then I think everything's fine. 9 Q. It is fine, but it doesn't allow statements with regard to 10 the accuracy of account balances, does it? 11 A. Well, it's fine. It's just irrelevant. I mean it's 12 irrelevant to the real target population, which is all of the 13 account holders. 14 Q. And that's all the past and present individual Indian trust 15 beneficiaries, is that your understanding? 16 A. That's my understanding. 17 Q. Does the target population sample allow NORC or the 18 defendants to make a statement about the account balances for 19 its new target population? 20 A. Not for the account balances. It's still going to be 21 limited to account transactions. 22 Q. And so notwithstanding that, you don't have any 23 disagreement with what Dr. Hinkins stated, correct? 24 A. Not in this regard. If you correctly limit the 25 extrapolation to the population from which you've sampled, 1446 1 that's sound statistical reasoning. 2 Q. Now, Dr. Hinkins also stated in her rebuttal that the 3 qualitative meta-analysis report is not central to the sample 4 design; is that correct? 5 A. That's my recollection. 6 Q. What is your opinion in that regard? 7 A. I think it's appropriately relegated to not being central. 8 It doesn't provide a reasonable basis, so that's the right spot 9 for it. 10 Q. Is there any other basis for Dr. Scheuren coming to the 11 conclusion that you've seen in the record that the paper ledger 12 era is similar to the electronic ledger era, other than a 13 meta-analysis? 14 A. Well, a meta-analysis I don't think is a reasonable 15 foundation to make that conclusion, and everything that I have 16 reviewed is to the contrary. 17 Q. With respect to your overall views of what is being 18 prepared here, and that is -- the error rate is one of the most 19 important issues, isn't it? 20 A. It's absolutely fundamental to the entire discussion we're 21 having. 22 Q. Do you recall reading in the transcript testimony the 23 testimony of Dr. Scheuren with respect to the error rate in the 24 LRIS tract history reports? 25 A. If this is the DataCom issue, then I believe that's -- 1447 1 Q. Let me identify for purposes of this question a document 2 from the administrative record which has Bates No. 50-2-1. Now, 3 have you reviewed this document, Mr. Duncan? 4 A. Yes, I have. 5 Q. I'd like you to turn to Bates No. 8. I'd like you to focus 6 on the statement that is highlighted. It states, "We," which is 7 NORC, "particularly emphasize our positive results in view of a 8 report on the findings of the TIME project carried out by 9 DataCom in 2000. DataCom asserts an error rate of about 30 10 percent in postings. NORC has found no evidence of such a 11 figure." 12 Is this an example of the importance of a sound definition 13 of an error rate? 14 A. Yes. This goes on to say that "such stark differences with 15 our findings are not explained by statistical variance alone." 16 That's absolutely correct. This is definitional. If an 17 error -- if many of the things were defined as an error from the 18 standpoint of DataCom but not from the standpoint of NORC, 19 that's how you can get these kinds of stark contrasts. 20 Q. And the contrasts as a result of a narrowly defined error 21 rate provide meaningless results for a trust beneficiary, 22 correct? 23 A. It's my conclusion that the way these error rates have been 24 defined, it's so difficult to clear the hurdle of being an error 25 that the error rate premise underlying all the statistical 1448 1 analysis is unfounded. 2 Q. I would like to ask you if you agree on one final opinion 3 with something that Dr. Scheuren stated. And I'd like to pull 4 up for purposes of this question Bates 57-32-1. This is the 5 straw man pretest. Do you recall reviewing this document? 6 A. I do. 7 Q. First of all, do you have any understanding of what the 8 straw man pretest is about, based on your review of it? 9 A. My review of this document indicates to me that this is an 10 attempt to -- the notion of a straw man is to put up an argument 11 that can be fought. Generally speaking that argument is a 12 fallacy. It's meant to distract. But it's really just to put 13 something up that everyone can take shots at. 14 Q. And I'd like you to pay particular attention to page 2, 15 Bates No. 2. And the last sentence in the second paragraph. 16 Dr. Scheuren stated that he prepared this document, and it 17 states, "Clearly we are not talking about the standards used in 18 a court case." 19 Would it be your understanding that the standards that are 20 being used with regard to the statistical sampling are not 21 standards that would allow a trust beneficiary to receive an 22 accurate statement of his account balances? 23 A. That is precisely my opinion. 24 Q. Would it be your opinion that the standards used -- 25 THE COURT: Let him give his opinion instead of you 1449 1 stating it and asking him to say yes or no. I mean this is sort 2 of the ultimate question. 3 BY MR. GINGOLD: 4 Q. Explain your opinion. 5 THE COURT: Thank you. 6 THE WITNESS: The fundamental premise is what is the 7 reconciliation, has something been reconciled or not, and is it 8 an error or is it not. And at the end of the day the statements 9 that are going to be able to be made from the statistical 10 sampling that was done and from the reconciliation efforts that 11 were done will not provide results that really would have any 12 meaning to the individual account holder. 13 They don't have any understanding of whether or not their 14 balance is correctly stated. There's going to be some level of 15 assurance that the transactions are probably correct, but at the 16 end of the day, as far as providing information to the 17 individual account holder on their account, I would contend, it 18 is my opinion, that the information that's being prepared as 19 part of the 2007 Plan would be meaningless. 20 MR. GINGOLD: Your Honor, our cross-examination is 21 completed. I would like to move into evidence -- 22 THE COURT: I don't think the cross-examination has 23 begun yet. 24 (Laughter) 25 MR. GINGOLD: I'm sorry. We spent a week and a half, 1450 1 Your Honor, of cross-examination. I'm sorry. The direct 2 examination is completed. We'd like to move into evidence 3 Mr. Duncan's expert report and his expert rebuttal report. 4 THE COURT: All right. They'll be received. 5 (Plaintiff Exhibit Nos. PPX 6 4284, 4484 received into evidence.) 7 MR. GINGOLD: Your Honor, one more we would like to 8 try and move into evidence, which is the demonstrative 9 identified as PPX 4486, which is compiled based on the DCV 10 information provided in the September 30, 2007, FTI DCV report 11 and testimony from Ms. Herman. 12 THE COURT: That will be received. 13 MR. GINGOLD: Thank you, Your Honor. 14 MR. WARSHAWSKY: For the record, we object. It's 15 demonstrative. It's not substantive evidence. 16 THE COURT: It'll be received. It'll be part of the 17 record. It'll be received as demonstrative evidence. How does 18 that sound? 19 MR. WARSHAWSKY: Sounds perfect. 20 (Plaintiff Exhibit No. PPX 21 4486 received into evidence.) 22 THE COURT: All right. Mr. Warshawsky, you may 23 cross-examine. 24 MR. GINGOLD: Your Honor, there's one more, I'm sorry. 25 THE COURT: Somebody get the hook. 1451 1 MR. GINGOLD: Oh, I'm sorry. Okay. 2 THE COURT: No, no. Go ahead. I'm just kidding. 3 Obviously, you smiled, though, Mr. Gingold. You have a nice 4 smile. 5 MR. GINGOLD: Thank you, Your Honor. PPX 4468, which 6 is a document entitled "The Treasury Department's role in the 7 administration of tribal trust funds." It's a Morgan Angel 8 report dated April 16, 2004. 9 MR. WARSHAWSKY: Your Honor, we object again. This is 10 a draft document. It was prepared for Treasury. It's not 11 probative or relevant to the issues. 12 MR. GINGOLD: Your Honor, all Morgan Angel -- Ed Angel 13 testified that they routinely prepare their documents in draft 14 until they're otherwise told by the Defendants to produce them. 15 THE COURT: Do we know what this is? This is a draft. 16 Everybody agree this is a draft? 17 MR. GINGOLD: No, Your Honor. We do not. They 18 routinely dub their documents draft documents. The testimony 19 is, not only before this court in this current proceeding, but 20 in the trial 1.5, Mr. Angel repeatedly responded to questions of 21 Mr. Harper that they routinely dub all their documents draft 22 documents. 23 MR. WARSHAWSKY: Your Honor, as you may recall, this 24 document was the subject of a motion that we filed. This was 25 inadvertently disclosed in discovery, and most certainly is 1452 1 considered by Treasury and Morgan Angel to be a draft document. 2 With regard to tribal -- 3 THE COURT: We had Dr. Angel up here and nobody asked 4 him what this was. 5 MR. WARSHAWSKY: It wasn't relevant to our 6 presentation of Dr. Angel. 7 MR. GINGOLD: Your Honor, it was identified in the 8 opposition to -- in the motion to suppress this document, the 9 government identified this as -- 10 THE COURT: It will be part of the record of this 11 case, but it will be assumed to be a draft unless somebody has 12 some other testimony establishing to the contrary. 13 What about 4485, your demonstrative? 14 MR. GINGOLD: We would like to move all the 15 demonstratives in, Your Honor. 16 THE COURT: All right. I'll receive 4485 as a 17 demonstrative exhibit as well. 18 MR. GINGOLD: Thank you, Your Honor. 19 THE COURT: Thank you. 20 (Plaintiff Exhibit Nos. 4468 21 and 4485 received into 22 evidence.) 23 CROSS-EXAMINATION 24 BY MR. WARSHAWSKY: 25 Q. Good afternoon, Mr. Duncan. 1453 1 A. Good afternoon, Mr. Warshawsky. 2 Q. Mr. Duncan, you've testified about a wide variety of 3 subjects this morning and this afternoon. Is that fair to say? 4 A. I think they've all been pretty focused to the issue of the 5 2007 Plan and the sampling associated with it. 6 Q. I appreciate you saying that. Principally, you're here to 7 talk about statistics, right? 8 A. Well, statistics and also to address the issue of the DCV, 9 which was provided to me in late September. It was not provided 10 before we provided our expert opinions. But my understanding at 11 the direction of counsel was that I was to analyze that and that 12 the Court would listen to my opinions and my conclusions 13 regarding the DCV. 14 Q. And that's why you provided us with Plaintiffs' Exhibit 15 4486 last night, correct? 16 A. Correct. 17 Q. All right. We'll get to that one. Let's talk about your 18 background in statistics, please. Can you put up Plaintiffs' 19 Exhibit 4284, please? This is your expert report. You remember 20 this. And specifically go to pages 43 and 44 of 79. 21 Mr. Duncan, do you recognize these two pages? 22 A. I do. 23 Q. And what is set forth on these two pages? 24 A. This is my deposition and testimony history over the last 25 10 years, I believe. 1454 1 Q. And I counted it, it looked like there were about 32 2 matters in which you've testified? 3 A. Correct. 4 Q. And of those 32, roughly 20 since you testified in the 2003 5 Cobell case; is that correct? 6 A. That's correct. 7 Q. So, in other words, in the intervening 50 months since you 8 testified in the Cobell case, you've testified as an expert in a 9 different matter about every two and a half months; is that 10 correct? 11 A. Seems about right. 12 Q. And would you identify, putting aside the Cobell matter, 13 the matters on those two pages in which you've been admitted or 14 accepted as an expert on statistics. 15 A. The final piece there, Mirage Crossings, is a statistical 16 sampling report as it relates to construction defect cases. The 17 opposing expert on that case is a Ph.D. statistician out of 18 Berkeley, I believe. 19 Q. And you've been accepted by the Superior Court of Arizona 20 in that case as a statistician. 21 A. I'm sorry, I believe you asked me which ones I'd testified 22 on. So this is a testimony in deposition only at this stage. 23 Q. All right. Okay. Can you identify any where a court has 24 accepted you to provide opinions as a statistician? 25 A. If you can zoom out and we go up a little bit. 1455 1 Q. I know you brought some papers up there. Do you have a 2 copy of your report up there by chance? 3 A. I believe I may. 4 Q. With the caveat that if there are notes in there I might 5 want to look at them. If it would assist you to review those. 6 A. This is a copy of my expert report and, given your caveat, 7 I will tell you that there are no notes. 8 Q. Okay. So feel free to refer to that if that would assist 9 you. 10 A. In the interest of trying to keep my carry-on baggage 11 light, I have omitted my resume. I'm pretty familiar with it 12 from the standpoint of needing to review it on the plane, so I 13 don't make a copy of it. 14 Q. My question was, though, whether you had been admitted as a 15 statistician in any of these matters? 16 A. Well, let me go back and look on the screen here. Go down, 17 please. I'd have to go back and look at both -- what's called 18 B&H Manufacturing, Inc., whether or not I was specifically 19 tendered as a statistician in that context, but there was quite 20 a bit of statistical testimony there. And in the Mitsubishi 21 versus Hughes Electronics case. I'd have to look whether that 22 was specifically -- I was tendered to the court in that role, 23 but that was a very central part of both of those testimonies. 24 Q. You don't remember that you were actually admitted as a 25 statistician there? 1456 1 A. I don't recall either way. 2 Q. In your selected experience summary, pages 35 to 40 of your 3 expert report, you listed 16 separate areas of experience. 4 You're familiar with those 16 areas, of course. 5 A. Yes, I am. 6 Q. Have you listed being a statistician in any of those 16 7 areas? 8 A. A lot of those areas obviously contain statistical 9 analytical work, but I have not listed straight -- statistical 10 analysis is a criteria permeates really all of the analytical 11 work that I do. 12 Q. You know, I asked you that specifically in the 2003 13 hearing, and even after testifying in the 2003 hearing, you 14 didn't update your CV to list statistics now as being one of 15 your professional experience areas, correct? 16 A. I didn't think the criticism was relevant then either. 17 Q. The answer to the question, though, is you did not update 18 your CV to say you're now a statistician. 19 A. That's correct. The statistical work that I do as embodied 20 in some of these representative cases, I don't feel it necessary 21 to clarify for the individuals that I provide my resume to, 22 beyond the fact that there's a significant component of 23 statistical work, I don't feel required to try and split that 24 out as a separate category. It permeates. 25 Q. Okay. Fair enough. Pages 40 and 41 of your expert report, 1457 1 you describe your teaching experience, and you list four areas 2 of economics. Do you remember that, sir? 3 A. Yes, I do. 4 Q. At the bottom of this page, continues on to the next page. 5 And in none of those instances do you even refer to statistics 6 as being an area of your teaching, where you teach, correct? 7 A. Again, as part of the micro and macroeconomics, there is a 8 significant component which is -- and you asked me previously in 9 my deposition if there were statistics contained in those 10 courses, and I informed you there was statistics, sampling, 11 sample design construction. Those are part of those courses 12 that I was teaching. 13 Q. But it's not a subject that you actually teach, statistics? 14 A. Separate and apart from economics, I do not. It's part of 15 the statistical analyses and survey work that I've done as an 16 economist. 17 Q. Okay. And we've got up here on the screen page 41, the 18 publications that you've authored. Any of those have to do 19 strictly speaking with statistics? 20 A. All of these would in some part have a statistical 21 component to them. A lot of the economic blue chip forecast 22 work that we do is premised upon statistical analysis work that 23 I'm doing of trends, macroeconomic and microeconomic trends. 24 The notion of intellectual property and antitrust litigation. A 25 lot of those underpinnings have statistical analytical work 1458 1 involved in them as well. 2 Q. But for example, you haven't authored a publication in a 3 statistics -- a professional statistics journal. 4 A. I haven't authored a publication in an economics journal 5 either. As an economist, working as an expert witness, that is 6 not something I regularly engage in. 7 Q. That's your profession, by the way, isn't it, being an 8 expert witness? 9 A. I would say that's probably 80 plus percent of the work 10 that I do. 11 Q. Now, one of the things that you disclosed on your CV was 12 that you're a chartered financial analyst. What's a CFA? 13 A. It's a chartered financial analyst. 14 Q. Okay. And for those of us who aren't CFAs, can you tell us 15 what that means? 16 A. It is -- a CPA is an application of an undergraduate degree 17 in accounting. Usually there's an accreditation occurred with 18 using those skills as an accountant. The CFA would be the 19 equivalent of that except it's an application of graduate level 20 skills in economics, finance, statistics, real estate, 21 securities markets and bond markets. 22 Q. Principally involved with investment management, investment 23 banking, financial analysis of stocks, bonds, correct? 24 A. No. That's a big part of it, but it's not principally 25 that. There's also a significant valuation component built into 1459 1 that. Valuing closely held businesses, things of that nature. 2 Q. It's not a title that one associates with a statistician, 3 right? 4 A. Several of my colleagues that are involved and hold the CFA 5 charter are statisticians. They have graduate degrees in 6 statistics and a lot of the analytical work they do as CFAs, 7 particularly underlying big bond portfolios and stock 8 portfolios, like pension funds, the work they do day to day is 9 as a statistician, but it's applied in the field of finance. 10 Q. You're talking about colleagues that have graduate degrees 11 in statistics, correct? 12 A. Correct. 13 Q. You don't have a graduate degree in statistics, do you? 14 A. That's correct. 15 Q. In fact, on page 42 of your expert report you describe your 16 educational background. Master's and a bachelor's degree in 17 economics, and your master's degree you list, what is it, five 18 areas of specialization? 19 A. Correct. 20 Q. And none of them are statistics, right? 21 A. No. As we've talked about in my deposition years ago, 22 econometrics is specifically the application of statistics and 23 statistical concepts in the field of economics. So I would say 24 that it is exactly that. It is statistics, just as economists 25 use it. 1460 1 Q. Well, just to cut through it, there are people who use 2 statistical analysis in their work, and that's you, right? 3 A. I certainly do. 4 Q. And there are statisticians like Dr. Scheuren and 5 Dr. Hinkins, right? 6 A. I don't know that you make a distinction. I think they 7 certainly use statistics in their work as well. 8 Q. You consider yourself to be an equivalent statistician to 9 Dr. Scheuren and Dr. Hinkins? 10 A. I don't have a Ph.D. in statistics, but from the standpoint 11 of using it on a day-to-day basis, I would say that we do very 12 similar things. 13 Q. By the way, I remember at the 2003 hearing, you joined the 14 ASA, the American Statistical Association in the middle of the 15 trial. Do you remember that, sir? 16 A. Correct, I did. 17 Q. How's that membership going? 18 A. I've let that membership lapse. It really was not very 19 beneficial. 20 Q. So you didn't go to their meeting last year in Salt Lake 21 City? 22 A. I did not. 23 Q. Now, let's talk a little bit about some of the stuff you 24 have done as an expert witness. Obviously, today you testified 25 about issues involving statistics. You've testified about a lot 1461 1 of other areas beyond just what a statistician does, or even 2 economists, right? In 2003, for example, you offered opinions 3 to Judge Lamberth about the legal application of Daubert in 4 Kumho Tire, do you remember that? 5 A. I provided testimony as it related to whether or not the 6 expert work that was done by some of the other experts in the 7 case was -- properly relied upon other experts, which I opined 8 that it did, and the use of geographic information systems, as 9 that was something that was typically relied upon by experts, 10 and in my experience I provided an opinion to the court that it 11 was in fact something that was used regularly by experts. 12 Q. In fact, it was your opinion, though, for Judge Lamberth 13 that the analysis of Mr. Fasold and the folks who worked for 14 him, that that analysis satisfied the reliability and relevance 15 standards of Daubert in Kumho Tire, correct? 16 A. Correct. 17 Q. And you're not an attorney, are you? 18 A. I'm not. 19 Q. That may not be a bad thing. But let me ask you, page 43 20 of your expert report, you talk about a case that you worked on 21 called Karron Power versus Delta Houseboat. Can you describe -- 22 that's in the U.S. District Court for the Eastern District of 23 California, right? 24 A. Correct. 25 Q. What was your opinion -- the subject matter for your 1462 1 opinion there? 2 A. This had to do with the quantification of damages to 3 Ms. Power. She was a surgeon and she had actually had her 4 finger severed in a boating accident, and it was an analysis of 5 the statistical information on projected future earnings, 6 projected for her practice, had to look at Bureau of Labor 7 Statistics, what likely her profession would be and what 8 progress, path she would make going down that road, and quantify 9 her personal injury loss as a result of that accident. 10 Q. And the opinions that you offered in that case actually 11 extended to opinions about vocational rehabilitation and life 12 care issues, right? 13 A. On a very limited basis was I involved in the life care 14 side of that. There were -- some of those inputs were required 15 for quantifying her economic loss as a physician, as a surgeon. 16 Q. In fact, your report was the subject of a motion in limine 17 because you had offered opinions about vocational rehabilitation 18 and life care issues, right? 19 A. It certainly was. 20 Q. And why don't you go ahead and pull that up, please. This 21 is the first page of the motion. Do you recognize this, sir? 22 A. Yeah. It's been some time since I've seen this, but yes. 23 Q. Specifically I want to ask you about something. Would you 24 go to page 30 of 69, and continue to the next page. Now, that 25 looks familiar. Can you -- do you recognize what that document 1463 1 is? And actually, we can go through it if you would like. 2 There's about 39 pages here. Why don't you flip through a few 3 pages, see if Mr. Duncan recognizes it. 4 A. I do. 5 Q. What is this? 6 A. This is my expert report in that case. 7 Q. Would you go to pages 53 to 55 of the document, please. 8 What's set forth in these three pages? 9 A. These are a listing of, at that time, all cases where I had 10 been retained as an expert witness, issued an expert report, and 11 then in addition the deposition testimony and trial testimony 12 columns that we saw previously in my expert report in this case. 13 Q. And so if you want to flip back, you can see I've put some 14 red arrows on the left-hand side. These were matters where I 15 didn't see a corollary -- you know, a listing for those cases in 16 your expert report in this case. I gather these are the matters 17 where you didn't testify but where you had rendered an expert 18 report; is that correct? 19 A. Correct. It either settled or otherwise been resolved 20 themselves before -- 21 Q. I beg your pardon? 22 A. They had either settled or resolved themselves before I 23 provided expert testimony. 24 Q. And if I'm correct, I counted that you provided expert 25 reports or testified in about 29 matters in the 50 months since 1464 1 you last testified in this case. Does that sound about right? 2 A. I think that number is awfully low. This was all the way 3 back in 2005 and we're in 2007 now. 4 Q. Good point. So it's more than that? 5 A. Yeah. I would think it would be significantly more than 6 that. 7 Q. So basically you issue an expert report or testify in a 8 matter about every, what, month, month and a half? 9 A. Probably average. Maybe a little bit more than that, but 10 yeah. 11 Q. Why don't you refer to page 44. Let's go back to the 12 expert report in this case. So we're back to Plaintiffs' 13 Exhibit 4284. And you're going to have to blow this one up, 14 obviously, but I'm looking at the Carlos Huerta matter. This is 15 a matter in the District Court, Clark County, Nevada. Do you 16 see that? 17 A. Yes. 18 Q. What were you retained to provide opinions about in that 19 case? 20 A. There were a couple of different opinions in that case. 21 One was quantification of the damage that -- Mr. Huerta was a 22 real estate broke r and he claimed that -- well, he actually 23 bought the rights to this lawsuit from another person, but he 24 was a real estate broker and he was suing Horizon Ridge Medical 25 Plaza for alleged breach of contract associated with selling a 1465 1 property upon which they were going to build a medical office 2 building. 3 Q. Well, you also provided opinions about whether a 4 transaction was a fraudulent conveyance, right? 5 A. Correct. That was the second of the two opinions. 6 Q. Why don't you tell us about that opinion. 7 A. This was simply an analysis of whether or not Horizon Ridge 8 Medical Plaza -- the owners of Horizon Ridge Medical Plaza made 9 a distribution to the owners at one point in time, and there was 10 a claim brought in the course of the other litigation, there was 11 a claim brought that that transfer in fact made Horizon Ridge 12 Medical Plaza insolvent. So as a follow-up report I was asked 13 to analyze whether in fact that transfer was a fraudulent 14 transfer and whether it resulted in them being insolvent. 15 Q. And what kind of expertise did you apply in making that 16 determination? 17 A. Again, basic economic finance skills, as we were looking at 18 the financial statements. As a chartered financial analyst, I 19 frequently look at financial statements to try and understand -- 20 I've testified in bankruptcy court before on the soundness of 21 plans presented to the bankruptcy judge. So here I specifically 22 was looking at whether or not that transaction rendered the 23 company insolvent. 24 Q. Can you pull up the expert opinion in that matter, please. 25 Mr. Duncan, I've had this put up on the screen now. Does this 1466 1 look familiar, sir, what's up here? 2 A. Yes. 3 Q. What is this? 4 A. This is my second expert report, the one that dealt with 5 the fraudulent transfer issue. 6 Q. Okay. And this, you stated that you concluded "Defendants' 7 actions did not wear a significant number of the badges of fraud 8 to constitute an intentionally fraudulent transfer." Badges of 9 fraud, is that a legal concept? 10 A. It is. It's articulated in the Nevada revised statutes. 11 Q. And so you were rendering another legal opinion. Is that 12 correct? 13 A. That's not correct. That was specifically a topic of a 14 motion in limine to have this opinion excluded, which was also 15 unsuccessful. The judge deemed that it was appropriate for me 16 to opine to the financial component of those badges of fraud, as 17 it related to the claim that it was an intentionally fraudulent 18 transfer. Many of those badges turn on specific issues, like 19 how much net equity does the company still have after the 20 transfer, and those were squarely within my skill set as an 21 economist and a financial expert. 22 Q. You said this was the subject of a motion in limine that 23 was also not allowed; is that correct? 24 A. Correct. 25 Q. Was the motion in limine in Karron Power ruled on? 1467 1 A. I don't recall that specifically, but as far as I'm aware, 2 I've never had a motion in limine to exclude my opinion accepted 3 by the court. 4 Q. Well, you didn't mean to suggest that the Karron Power 5 motion had been denied; is that correct? 6 A. I'm not aware one way or the other. I ultimately -- I 7 believe that case settled. I believe I testified at deposition 8 perhaps, and then the case settled. 9 Q. Let's go to page 44 of your expert report, please. 10 THE COURT: Mr. Warshawsky, are you going to bring 11 this man up on charges of unauthorized practice of law? 12 MR. WARSHAWSKY: No, I'm actually going to get to 13 the -- 14 THE COURT: Then let's get on with the statistics part 15 of this. 16 MR. WARSHAWSKY: Okay. Very good. I do have to ask 17 Mr. Duncan about one matter he testified in very briefly, 18 because it does relate to some of the opinions he's offered 19 today. 20 BY MR. WARSHAWSKY: 21 Q. The Harbor Breeze matter. That was one before the 22 California Public Utilities Commission earlier this year? 23 A. Correct. 24 Q. And your opinions were rejected in that, right? 25 A. The application was rejected. 1468 1 Q. Well, in fact, didn't the PUC find that your assumptions 2 were unrealistic? 3 A. I testified in front of the Public Utility Commission. I 4 did not review their findings at the end of the day. I 5 understand that they did allow this other party to come in and 6 begin to provide ferry service back and forth to Catalina 7 Island. But as far as what the basis of the specific 8 conclusions were, I'm not aware. 9 Q. Well, you had made an assumption that, what was it, five 10 trips a day or something like that in your analysis? 11 A. My analysis went to an estimation of how would the existing 12 ferry service -- very simple notion. The ferry service, they're 13 given a limited monopoly power to operate both during the summer 14 and the winter. They make lots of money during the summer. 15 They don't make much money during the winter because there are 16 not as many people going out to Catalina Island. 17 And my task was, in meeting with the chamber of commerce 18 and the mayor in the city of Catalina, to try to come up with an 19 estimate of, if someone else was allowed to come in and operate 20 during the summer -- sorry, I was trying to speed up for the 21 Judge. 22 THE COURT: You can't speed up fast enough for the 23 Judge. 24 BY MR. WARSHAWSKY: 25 Q. I'll take the cue there. Let's go straight to Plaintiffs' 1469 1 Exhibit 4486, then. Actually, you all have that. This is 2 Plaintiff's Exhibit 4486 that you testified about at length 3 today. Let me ask you, first of all, when you prepared -- when 4 did you prepare this chart, by the way? 5 A. We finished last night. 6 Q. When did you start working on it? 7 A. Saturday morning. 8 Q. And who worked on it? You worked on it, I gather? 9 A. I did, as well as two of my staff. 10 Q. Now, when you prepared the chart, did you study the entire 11 DCV report before reaching your conclusions there? 12 A. I read the original DCV and the DCV that was presented 13 September 30. I understand there were some additional documents 14 that were subsequent to that. I don't know that I've reviewed 15 those. 16 Q. Did you read the Alaska regional report in connection with 17 the DCV? 18 A. I believe so. 19 Q. Can you pull up DX 153A, 20. Actually, before we do that, 20 let me ask you, go back to 4486. Sorry, we're kind of making 21 this up as we go, because we did just get this ourselves. 22 Your missing months, column 7, how did you compile the 23 numbers shown under column 7 for the months of missing data? 24 A. There's a footnote there that explains that. We took the 25 Tab 2 regional reports for months prior to the IRMS conversion. 1470 1 And then for those six regions where we didn't have any 2 information, we took an average of the months that we had for 3 the regions that we did have. 4 Q. In other words, if a region didn't have data in December 5 1984, you counted it as a missing month, right? 6 A. Correct. 7 Q. And meaning that it didn't have data for the IRMS system, 8 right? 9 A. Correct. 10 Q. You assumed, did you not, that every one of those regions 11 converted to IRMS as of October 1984? 12 A. That was the implicit assumption in trying to estimate 13 whether or not there were missing months for those regions. 14 Q. And so if in fact a region converted after October 1984, 15 your assumption's erroneous, right? 16 A. It would potentially change the calculations. 17 Q. If, for example, Alaska converted around February of 1987, 18 you would have counted about, what, two and a half years of 19 missing data that wasn't really there, right? 20 A. That would be correct. 21 Q. And you don't know, for example, when the Western Region 22 converted, do you? 23 A. I believe that information is in the DCV. 24 Q. Well, if I told you between October 1985 and March 1987, 25 does that sound consistent with what you recall? 1471 1 A. Roughly. 2 Q. Okay. So to the extent your model assumed that the Western 3 Region converted in October 1984, you've overstated the number 4 of months of missing data there as well, right? 5 A. That's correct. 6 Q. And if I told you that the Pacific converted between March 7 of 1986 and November of 1986, again you've overstated the 8 Pacific. 9 A. There might be 12, 15 months of data there where we don't 10 know whether or not they had converted, or if we did find out 11 they had not yet converted, that would be included in this. 12 Q. By the way, back in the Western, if in fact Fort Yuma 13 didn't convert until 1996, you've assumed about 12 years of 14 missing data that really wouldn't be there because they weren't 15 part of IRMS until 1996, right? 16 A. That would potentially be correct. 17 Q. Potentially be correct. How would it not be correct? 18 A. You have to go back and look specifically to see whether or 19 not any of these regions had been on essentially other previous 20 electronic systems, or if this was their first conversion to 21 IRMS. Because I know that many of these actually were on 22 electronic systems well before the time we are talking about the 23 conversion to IRMS. They were on the systems all the way back 24 to the 1970s. So depending on how that exactly is being 25 accounted for on the DCV, it may or may not overstate the number 1472 1 of months of missing data. 2 Q. Why don't you pull up Exhibit 153A. Let's go to page 20. 3 Mr. Duncan, did you review this particular page when you were in 4 the process of preparing Plaintiffs' Exhibit 4486? 5 A. I believe that I have reviewed this. 6 Q. And would you read the fourth and fifth paragraphs? That's 7 actually the paragraph above the highlighting, and the 8 highlighted paragraph. You can just read that to yourself. 9 (Witness reviewing document.) 10 A. Okay. 11 Q. Did you consider that when you concluded that Alaska had 16 12 months of missing data? 13 A. I don't recall looking at that specifically as we were 14 trying to pull together what the flip side was of what 15 Ms. Herman had testified, how much they had gotten done, I don't 16 recall specifically looking at this and trying to calculate how 17 much was left to be done. 18 Q. Well, if you had looked at that, would that cause you to 19 reconsider your conclusion that there were 16 months of missing 20 data for Alaska? 21 A. Yeah. It may. Again I'd have to go back and look and see 22 if Alaska had converted to another electronic system prior to 23 this. 24 Q. Would you pull up page 33 from 153A. Did you look at this 25 page when you prepared Plaintiffs' Exhibit 4486? 1473 1 A. I don't recall looking at that specifically. 2 Q. Do you know what that means, by the way, when it says all 3 accounts in the region rolled forward? 4 A. This is what we talked about earlier today of their 5 accounts being out of balance. If one account from the ending 6 month doesn't match on the opening of the next month, then 7 you've got -- potentially have an issue that it hasn't been 8 rolled forward correctly. 9 Q. Was that an issue in this case? 10 A. It is not. It would surprise me if it was too big an issue 11 in Alaska, because they were fairly recent in coming on to some 12 of these systems. 13 Q. And had you reviewed this particular page, would that alter 14 your position about 16 months of missing data for Alaska? 15 A. Again, I think as I've already answered, I'd have to go 16 back and look and see, make sure there wasn't another system 17 they had transferred onto prior to this. 18 Q. Let's go back to 4486, please. Another assumption you 19 made, I believe -- I assume anyway in preparing this chart, is 20 that an agency or region would have transactions every single 21 month; is that correct? 22 A. Correct. 23 Q. And in fact, do you know if that's the case? Were there in 24 fact some agencies, agency offices that had months where they 25 didn't have transactions? 1474 1 A. It may be possible. To the extent that there were months 2 where agencies didn't have transactions, that should be 3 accounted for in the weighting, of going back and looking over 4 all the months we do have information. If there were instances 5 where there were not transactions, it would drag down the 6 average number of transactions per month or the average dollar 7 amount per transaction. 8 Q. But if an agency office, for example, didn't process 9 transactions in a month, you treated that as a missing month, 10 right? 11 A. That's correct. Well, let me back up. It is treated as a 12 missing month with respect to what's reported in the DCV, and 13 that was the basis upon which we did the calculation. The 14 missing month column. 15 Q. Did you read Ms. Herman's testimony in this matter? 16 A. Yes, I did. 17 Q. Do you remember reading the cross-examination when she was 18 asked about whether agencies -- whether there were any agency 19 offices that regularly processed transactions on a monthly 20 basis? 21 A. I recall there being uncertainty as to whether or not some 22 of those missing months were potentially due to them not 23 processing transactions, but that that had not yet been resolved 24 definitively. 25 Q. Well, if in fact your assumption in this chart is 1475 1 incorrect, that an agency office processed transactions every 2 month, if that's wrong, your chart again overstates the number 3 of missing months data, right? 4 A. You know, whether or not it had been processed during that 5 month, you'd have to know whether or not when, the month that it 6 did get transferred, did it -- were the transactions that were 7 processed in the missing month processed in another missing 8 month's transactions, because it may be that they lumped them 9 all the next month. If that month is missing, then the way that 10 we've done the calculation, it would average out across both 11 those months. 12 Q. Since you mentioned averaging, you used, what, average 13 dollar per transaction; is that right? I seem to remember that 14 somewhere in here. Is that right? 15 A. Yes. 16 Q. Which column was that? 17 A. 4. Average throughput per transaction. 18 Q. In fact, isn't it true that there were some transactions 19 that were very large in dollar amount? 20 A. Correct. 21 Q. And many, many small transactions? 22 A. Correct. 23 Q. Doesn't it overstate the impact in your chart to use an 24 average for the transactions? 25 A. I'd love to have the information available to me to be able 1476 1 to do the other calculation, but with the information we had, it 2 was limited to being able to do this analysis. So it depends on 3 how the weighting works out between the number of transactions. 4 It's not part of the DCV that we can tell. 5 Q. And one other thing, going back to the 1980s, do you know 6 were there in fact -- what was the practice in terms of -- do 7 you remember whether there was a practice in terms of paying 8 interest every six months or whether interest was posted on a 9 monthly basis? 10 A. I may have known that at one point in this case, but I 11 don't recall now. 12 Q. So if in fact early on, for purposes of your analysis, 13 interest was posted every six months instead of every month, 14 again your chart would overstate the impact of missing 15 transactions, right? 16 A. It may potentially. You'd have to go back and look and see 17 how they actually broke out between the months. Again, data we 18 don't have access to. 19 Q. Okay. All right. In your report, let's go to 4284, 20 please. Now, you used an expression numerous times in this 21 report, you talked about Interior's plan -- we can look at page 22 5, for example. It's in the first paragraph, the heading right 23 at the top. Just do the paragraph. 24 See right there in the first sentence you talked about -- 25 used this expression about whether something, quote, "would 1477 1 likely confuse and mislead the layperson." 2 A. Correct. 3 Q. What kind of -- what field were you using or what expertise 4 were you using in reaching those conclusions? Is that a 5 statistical judgment? 6 A. It is. As a statistician and certainly as an expert 7 witness, I think when you're providing testimony to a jury or to 8 the judge you need to take into account who that audience is. 9 And when you provide information, you need to present it in such 10 a way that it wouldn't mislead that consumer of the information. 11 Q. Okay. And are there any standards in statistics you apply 12 in reaching that conclusion? 13 A. No, but based on years of experience, you can have some 14 idea as to who your audience is and whether or not they will 15 correctly interpret and use the information you're providing, or 16 whether you need to be more clear and not use statistical 17 jargon, but you need to be clear and make sure that it doesn't 18 potentially mislead someone. 19 THE COURT: Mr. Warshawsky, for your information, and 20 I guess for the information of all counsel here, I've received 21 this witness's report in evidence, but as you may know, in jury 22 trials it is not usual to receive an expert report in evidence. 23 An expert report is handed over to the other side so they can 24 cross-examine it. But what counts is the opinions rendered in 25 open court. 1478 1 This witness has testified in open court about his opinions 2 about irrelevant sample design, sample selection problem, et 3 cetera. He hasn't said anything until you raised the subject 4 about confusing and misleading the layperson, and I'm not going 5 to take that into account. You can cross-examine it if you'd 6 like, but what's written in his report means nothing to me. 7 MR. WARSHAWSKY: I would like to use, Your Honor, with 8 your indulgence, I would like to use the executive summary in 9 his report as a basis for talking, organizing the opinions that 10 he talked about today. 11 THE COURT: You can organize it any way you want to. 12 I just want to give you a heads-up that I don't care about 13 what's written in this report unless he's also testified to it 14 in open court. 15 MR. WARSHAWSKY: Very good, Your Honor. 16 BY MR. WARSHAWSKY: 17 Q. You testified quite a bit about sampling, but in fact, I 18 believe you actually frequently stated that you lack the 19 information necessary to evaluate the sampling plan that 20 Interior's put out, right? 21 A. I think I've testified that there's information about the 22 reconciliation activity that was clearly lacking. The 23 information that we had I thought formed a reasonable basis for 24 me reaching the conclusions about the sampling plan, 25 particularly then as I was able to read the information provided 1479 1 in the rebuttal reports. It was very clear to me what the 2 intentions and what the design was originally constructed to do. 3 Q. Okay. Let's look at the first bullet point here under the 4 executive summary. You talked about missing and destroyed data, 5 drawing from a target population not being possible. You 6 testified at length about that today, right, sir? 7 A. Correct. 8 Q. And isn't it true that the mere fact that some data is 9 missing or has been destroyed, that doesn't mean that there are 10 no other copies of that information available, right? 11 A. Doesn't necessarily mean that there are not other copies. 12 It doesn't mean that there are. It simply means there have been 13 data that are destroyed or missing. 14 Q. Have you reviewed any of the testimony in this case 15 discussing the redundancy of information within Interior's 16 system? 17 A. I recall that being in your opening comments. I don't 18 recall specifically other testimony that related to 19 redundancies. You may refresh my recollection. 20 Q. Well, did you read Dr. Scheuren's testimony? 21 A. I did. 22 Q. You don't remember him talking about redundancy? 23 A. It's not jumping off the page at me. 24 Q. Okay. Is it your opinion that it's impossible to take a 25 valid sample unless you can confirm that the sampling frame 1480 1 duplicates the target population? 2 A. We went around and around on this subject years ago in my 3 deposition. But the notion -- you can take a sample from 4 anything you want to take a sample from. The notion is what can 5 you do with it when you're done. So you're asking me is it 6 impossible to take a sample? No, you can take the sample. It 7 simply limits what you can say with the information from that 8 sample. 9 Q. Well, my question for you -- and Judge Robertson wasn't in 10 our deposition years ago, and he wasn't at the 2003 hearing. My 11 question for you is is it your opinion that the only way you can 12 take a valid sample is if the sampling frame duplicates the 13 target population? 14 A. You can take a valid sample from anything you want to take 15 a sample from. The question is what can you do with that sample 16 once you're done. Can you say anything about a part of that 17 population that was not available to draw a sample from? It is 18 my opinion that no, you cannot. 19 Q. Well, why don't you pull up the Reference Manual on 20 Scientific Evidence, Second Edition 2000. Mr. Duncan, are you 21 familiar with this document? 22 A. I am. 23 Q. Would you agree that it's a learned treatise with regard to 24 the various subjects covered within that report? 25 A. I think generally speaking it is. 1481 1 Q. Okay. And if you go to page 91 of the document, please, 2 this is the reference guide on statistics. Have you seen this 3 before? 4 A. I have. 5 Q. Now, would you agree that this is a learned treatise on the 6 application of statistics? 7 A. Again, I would say generally it is. 8 Q. Go to page 240, please. I'm sorry, it's actually not that 9 page. It's page -- we're going to have to bump it forward. 10 There we go. The one that has the highlighting. Are you 11 familiar with this section? 12 A. I am. 13 Q. In fact, we talked about that four years ago, right? 14 A. Correct. 15 Q. What is the question being posed about the sampling frame 16 approximating the population? What's your understanding as an 17 expert? 18 A. It really goes right to this issue of have you constructed 19 your sample, the sampling frame, which is how you're going to 20 draw the sample, have you done it in such a way that you can 21 approximate what the population is that you would like to make a 22 statement about. 23 Q. And you see the last two sentences that I've highlighted 24 there: "Frequently, however, the target population includes 25 members who are inaccessible or who cannot be identified in 1482 1 advance. As a result, compromises are sometimes required in 2 developing the sampling frame." That's what we're talking about 3 here, right? 4 A. Correct. 5 Q. So at least according to this treatise, you can go ahead 6 and draw a sample even if there are inaccessible or missing data 7 elements, right? 8 A. We agree on this point. You can draw the sample. The 9 compromise that's talked about here is whether or not you're 10 going to be able to make any sort of statement about something 11 that was not available to be sampled from. 12 Q. Okay. And you're aware from your review -- and by the way, 13 in your expert report -- I'll tell you, I didn't go through it. 14 I think you indicated you reviewed just about everything in the 15 administrative record, right? 16 A. When I prepare my expert report, to be the state court, 17 federal court, or U.S. District Court, I prepare them in 18 compliance with the federal rules of civil procedure, and that 19 would require me to list every document that we have received, 20 also called documents considered. I don't limit it to documents 21 relied upon. So if we received a document, we listed them all, 22 whether or not we reviewed them. 23 Q. Oh. So even though in your listing of documents you 24 considered, you may not have actually reviewed documents that 25 you listed as having been considered? 1483 1 A. Correct. My understanding is that the federal rules 2 require you to list everything that has been received, whether 3 or not you reviewed it, or whether or not you've relied upon it. 4 Q. You obviously reviewed materials regarding efforts to 5 restore transactions. The DCV, you've talked about that, right? 6 A. Correct. 7 Q. Aren't those the kinds of efforts that were being discussed 8 in that treatise about attempting to take steps to fill those 9 data gaps? 10 A. Certainly. That's one of the steps that would be 11 anticipated when you have not been able to sample from a target 12 population that you want to make an inference about, you can go 13 back and see if there's some way to identify why those 14 transactions in this instance were missing, and then you can 15 draw subsamples of those to try to see whether or not they're 16 different. 17 Q. You talked about this. Why don't we go ahead and pull up 18 Plaintiff's Exhibit 4485. This is your demonstrative. Go to 19 the third page. This is also, I believe, depicted in your 20 expert report, right? 21 A. Correct. 22 Q. Now, in the second box there, it looks like the recorded 23 transactions is roughly one fourth of the total population; is 24 that correct? 25 A. Seems about right. 1484 1 Q. Do you have any factual basis for telling Judge Robertson 2 that in fact the sampling frame in this case is about one fourth 3 of the target population? 4 A. I don't think anybody knows how big that missing and 5 omitted component is. 6 Q. Okay. So as far as you're aware, the sampled population 7 and the target population may actually be fairly close in size. 8 We know they're not identical, but close? 9 A. I would have no basis, nor have I seen anything in the 10 administrative record that indicates what the relative size of 11 those two boxes is. 12 Q. So Judge Robertson, when he looks at this diagram, 13 shouldn't draw any conclusions as to the size of the, relative 14 size of those two boxes; is that correct? 15 A. No. I perhaps should put on the bottom "not necessarily to 16 scale." We don't know. 17 Q. Okay. You indicated in your direct examination that you 18 reviewed the September 30, 2005, NORC report regarding the 19 reconciliation of the national sample and the high-dollar items. 20 This was AR-438, Bates No. 40-00002. Let's try to find it maybe 21 in the administrative record. AR-438. 22 And in reaching your conclusions about missing or destroyed 23 data, the impossibility of doing the reconciliation work, you 24 considered pages 12 to 17 of the report. 25 Why don't you show Mr. Duncan 12 through 17. 1485 1 Does this look familiar? 2 A. Yes, it does. 3 Q. And on page 13 of the report, table 4, how did you evaluate 4 the fact that the table indicates that all but nine out of 2,372 5 transactions had been reconciled? 6 A. This goes right back to the notion of the high, high hurdle 7 to be considered unreconciled. 8 Q. I'm sorry? 9 A. It goes back to the high, high hurdle to be considered 10 unreconciled. So here what it's saying is if it was accounting 11 code 1 or 2, which comprises these alternative procedures, it 12 would be considered -- it would be listed in the reconciled 13 section. But because we don't know what those alternative 14 procedures are, and they may or may not be directly reconciled 15 back to a supporting document but rather be subject to some 16 judgment of the accountants, the notion of reconciled here, as 17 I've testified to, would be meaningless to an individual account 18 holder. 19 Q. Well, you're a CFA but you're not a CPA, right? 20 A. That's correct. 21 Q. And so you're not here today to offer opinions about what 22 would be adequate reconciliation procedures, right? 23 A. That's correct. My understanding is Mr. Pallais will be 24 doing that. 25 Q. You talked about not knowing what alternative procedures 1486 1 were, right? 2 A. Correct. 3 Q. So you don't know if that's an auditor's concept? 4 A. I've reviewed the ASM to see whether or not it's something 5 they laid out in the ASM as an auditor's concept. I didn't see 6 anything there that made it clear what this notion of the 7 alternative procedure was. 8 Q. Okay. But speaking more generally, you don't know whether 9 auditors routinely talk about utilizing alternative procedures 10 in the course of doing audit work; is that correct? 11 A. I've encountered the term before in working with 12 PricewaterhouseCoopers. 13 Q. When you saw that term utilized in connection with the work 14 here, did it occur to you that "alternative procedures" was an 15 auditor's term of art? 16 A. I'm generally familiar with the term as it's been used by 17 accountants when I did work in helping design statistical 18 sampling when I was at PricewaterhouseCoopers, so I've heard 19 "alternative procedures" used in different contexts among 20 accountants. 21 Q. So basically you rejected the conclusion in table 4 about 22 reconciling all but nine of the transactions because of your 23 understanding of how errors were determined; is that correct? 24 A. Well, you're mixing two different concepts. You talked 25 about reconciled and you talked about errors. Reconciled is the 1487 1 first step. The second step is whether or not there's an error. 2 Q. All right. You rejected the conclusion that all but nine 3 transactions had been reconciled because of your understanding 4 of the standard required to reconcile a transaction; is that 5 correct? 6 A. I don't know that I rejected or discounted the notion. It 7 was reported here that it was reconciled. It was just my 8 opinion that given the hurdle associated with not being able to 9 be reconciled, that this doesn't hold any particular meaning to 10 the individual account holder. 11 Q. Okay. As a statistician, though, if it's reported to you 12 that all but nine out of almost 2,400 transactions was 13 reconciled, in your mind you would still consider there to be a 14 problem with missing data? 15 A. Well, this goes to how the error rates and the 16 reconciliation process is determined, and statisticians, as with 17 your clients, Dr. Scheuren and Dr. Hinkins, had input, they had 18 involvement in the discussion of what was reconciled and what 19 wasn't. So it matters because it then -- how one uses the 20 result of that reconciliation activity is premised in part, in 21 large part upon what the reconciliation means. 22 So as a statistician, I would view with great skepticism 23 the information that all but nine were reconciled. At a minimum 24 I would like to know how many were reconciled with a directly 25 supported document, how many were reconciled by this 1488 1 recomputation method, and how did that work. How many were 2 reconciled by third-party documentation. How many of these were 3 reconciled with the alternative procedures, and how was that 4 alternative procedure actually done. 5 Then I would be able to take this information and make some 6 kind of meaningful extrapolation up to the rest of the 7 population from which I drew the sample. 8 Q. And in the course of your rendering your opinions rejecting 9 these findings, you never reviewed any of the materials used by 10 the accountants to reconcile transactions, did you? 11 A. No, that's not correct. I've testified repeatedly I looked 12 at the accounting standard manual, which is the -- all the 13 different versions we were provided, there were at least two or 14 three we looked at because it's changed over time, but we looked 15 at the accounting standard manuals, and we looked at all the 16 information we did have, which was the Eastern Region study, but 17 we were not provided as part of the administrative record with 18 any of the actual source documentation where you specifically 19 tied one transaction back to another. 20 Q. And that's what I meant by my question. You never looked 21 at any of the stuff the auditors or the accountants actually 22 looked at in concluding that a transaction was reconciled, 23 right? 24 A. I looked for it and I didn't find it. 25 Q. Just to move it along -- oh, I'm getting a time-out sign. 1489 1 THE COURT: We'll take a 10-minute break. 2 (Recess from 3:42 p.m. to 3:56 p.m.) 3 MR. WARSHAWSKY: Your Honor, just from a planning 4 point of view, counsel were talking over the break. Does the 5 Court have a 4:30 matter, or is that gone? 6 THE COURT: We do but they're going to wait. 7 MR. WARSHAWSKY: Excellent. Let's see if we can get 8 through Mr. Duncan, then. Maybe you'll get your flight after 9 all. 10 THE WITNESS: I was just telling His Honor thank you. 11 BY MR. WARSHAWSKY: 12 Q. Just jumping back to one subject, you read Ms. Herman's 13 testimony? 14 A. I did. 15 Q. You recall her testimony about the Plaintiffs being 16 provided copies of the IRMS data in 1997? 17 A. I do recall that. 18 Q. In the course of your work, did you ever consider analyzing 19 the data that was provided to the Plaintiffs? 20 A. No, I have not. 21 Q. Now, let's go back to AR-438. Page 16, table 7. Do you 22 remember I was asking you before the break about the debit 23 transactions. I guess to shortcut it, credit transactions I 24 gather you would draw a comparable conclusion with regard to the 25 ability to reconcile the items in the national sample, missing 1490 1 data? 2 A. That's correct. Although here there are more differences 3 noted, but the same concept applies. 4 Q. In this case -- by the way, you remember what Dr. Scheuren, 5 what NORC did for those instances where they couldn't find 6 support for a transaction? 7 A. Well, you have to be more specific. What do you mean? In 8 which calculation? 9 Q. I will. Why don't you go back -- let's jump back, I'm 10 sorry, in the debit transactions. Okay. It's in the inferences 11 for debit transactions. Do you remember reading this section, 12 inferences for debit transactions? 13 A. Yes. 14 Q. Does that refresh your recollection as to how Dr. Scheuren 15 and the NORC team treated the debit transactions where they 16 weren't able to reconcile them? 17 A. Again, there's still the same two questions: How was it 18 treated from the attribute standpoint and how it was treated 19 from the variable standpoint. From the attribute standpoint it 20 was treated as an error, for the calculation of it being a 21 yes/no, was it an error. But for purposes of calculating it for 22 the variable, the dollar amount of the error, since it was 23 unreconciled, there was no information available, and the NORC 24 report assumed that that dollar difference was zero. 25 So it's some of both. It's an assumption about what the 1491 1 error rate is, it was treated like an error. For the dollar 2 difference, it was not. 3 Q. And what's the basis for your conclusion that that's how 4 they treated it for the variable analysis? 5 A. I'd have to go back and look at the documents. It's in 6 some of the detailed NORC reports, where they say there's a 7 couple of different ways we can deal with this when we don't 8 have information on the reconciliation. We could treat it as 9 the average dollar error for the reconciled transactions, where 10 there was an error, we could take that average dollar amount, or 11 we could treat it as a zero. And they've effectively treated it 12 as a zero. 13 Q. Are you talking about some of those draft documents and 14 some of those documents prepared, say like in the 2002, 2003 15 time frame? 16 A. I'd have to go back and look specifically where that was 17 at. 18 Q. You don't sitting here today remember any analysis done in 19 2005 that treated the unreconciled items as being not errors for 20 purposes of the variable analysis, do you? 21 A. Again, I'd have to go back and look at what date that 22 document was. 23 Q. And similarly, we can go to it -- by the way, Dr. Scheuren 24 says that, in the final paragraph up here, that the error rates 25 that he calculated, that those are based on a very conservative 1492 1 estimate, treating them as unreconciled, right? 2 A. For purposes of the attribute piece, yes. 3 Q. It's about as conservative as you could be, right? 4 A. No. I think Dr. Scheuren or Dr. Hinkins appropriately 5 indicated it would be an unsound statistical practice to do 6 otherwise. In her rebuttal report she specifically opined that. 7 I wouldn't describe it as being conservative. It's the right 8 way to do it. 9 Q. Thank you. I'll take that. Similarly for the credit 10 transactions, do you remember how the NORC team and Dr. Scheuren 11 treated unreconciled credit transactions? 12 A. My recollection is that there weren't any unreconciled 13 credit transactions. 14 Q. Why don't we jump ahead. Let's get to the inference for 15 credit transactions. This actually spills over to two pages. 16 There is this page right here, we're looking at this, what, 16, 17 the last paragraph on 16, where he sets out his percentages and 18 error rates and all that, and then spilling over to the next 19 page, go to the top of 17, you see the first full paragraph. 20 Does that refresh your recollection as to how the unreconciled 21 transactions were calculated? 22 A. No. I'd have to go back and look at what component of this 23 is unreconciled. This is just dealing strictly with what the 24 error rate is, the 4 percent error rate. You'd have to go back 25 and look and see what proportion of those were unreconciled. 1493 1 These are errors. 2 Q. I'm sorry, I'm looking at the next paragraph where he 3 actually quantified the $86 million, the 42 million. Do you see 4 that? 5 A. Yes, I do. 6 Q. Does that refresh your recollection as to how the NORC 7 analysis treated unreconciled credit transactions? 8 A. No, it doesn't tell you anything about what -- if there was 9 a difference, what that difference was. And on an unreconciled 10 transaction, it would be unclear to me what basis they would 11 have for what that difference was. It's not been reconciled. 12 Q. Okay. Let's go back to your expert report, second bullet 13 point on page 5, please. Why don't you blow that up. This is 14 the one, do you see that, about the sampled population is 15 different from the target population? 16 A. Yes. 17 Q. And you say that the plan to sample the target population 18 is conjecture, not statistical inference. Does your analysis 19 also take into consideration the work that the plan describes to 20 be completed with regard to things such as the DCV, the 21 land-to-dollars work, posting tests? 22 A. No, this specifically looks at the sampled population, 23 which was the subject of the sampling that was done, cannot be 24 inferred beyond the target population from which it was drawn. 25 Q. So you're not telling Judge Robertson that if Interior then 1494 1 goes ahead and performs the remaining work with regard to 2 posting transactions, land to dollars, the DCV work, that it's 3 not possible to draw valid statistical inferences about the 4 entire population here, are you? 5 A. Well, you would need to make sure that we don't have 6 missing and omitted transactions. And based on my review of the 7 documents, that is a hurdle that will not be cleared. As Arthur 8 Andersen opined, as the GAO opined, doing that kind of 9 reconciliation would not be possible given the paucity of 10 documents. 11 Q. Well, I understand that's your assumption. That's what 12 you're working from. My question for you, sir, is if Interior 13 does go ahead and performs the remaining work to be done which 14 is described in the plan -- and this is in the administrative 15 record if you want to look at it, we could pull that up, at 16 pages 17, 18 AR5-66. Do you want us to do that? 17 A. We certainly could, but what you would need to see is the 18 results of those, just as we've now, in 2007, have seen the 19 results of the sampling, you'd need to see the results of those 20 activities to opine whether or not it can be extrapolated to the 21 target population of all trust beneficiaries for all funds in 22 transactions and account balances. 23 Q. And at this point in time you are just not in a position to 24 tell Judge Robertson anything about those results because they 25 haven't happened yet, right? 1495 1 A. Well, I'm in a position to tell him that, in the first 2 instance, you're not going to be able to get to account 3 balances. It doesn't matter how much more sampling you do, 4 you're sampling transactions, so you're not going to be able to 5 get to -- when I read the 2007 Plan it's clear to me, it says 6 there would be accuracy and completeness with respect to the 7 account transaction listings and the account balan