But if selection by rank order of test scores can be shown to increase, to a statistically significant extent, the likelihood of improving the quality of performance, this Court is not empowered to mandate the change suggested by plaintiffs, however sensible that might seem.
After carefully considering the mass of testimony and exhibits in this case, I have concluded that the evidence as a whole slightly preponderates in favor of a finding that criterion-related validity has been established with respect to one or more (see individual findings above) of the criteria, in the sergeant and detective examinations, and a similar showing of content validity with respect to the corporal, sergeant, and detective examinations. In each case, however, the correlations are quite small, with the result that the utility of the examinations is relatively low.
The EEOC guidelines (§ 1607.5(c)(2)), while requiring that "in addition to statistical significance, the relationship between the test and criterion should have practical significance" also state that "the larger the proportion of applicants who are hired for or placed on the job, the higher the relationship needs to be in order to be practically useful. Conversely, a relatively low relationship may prove useful when proportionately few job vacancies are available" and that "the smaller the economic and human risks involved in hiring an unqualified applicant relative to the risks entailed in rejecting a qualified applicant, the greater the relationship needs to be in order to be practically useful. Conversely, a relatively low relationship may prove useful when the former risks are relatively high." The FEA guidelines counsel a similar approach.
In the present case, the selection ratio is extremely small. And, while I agree with plaintiffs that the defendants' argument tends to exaggerate the risk factor, I must nevertheless recognize that there are serious risks attending a failure to achieve a high level of performance throughout the Police Department.
None of the administrative guidelines referred to in this case have the force of law. They are, however, entitled to deference, by reason of the high degree of expertise which has entered into their formulation. While the EEOC guidelines mandate criterion-related validity unless that approach is technically infeasible, the FEA guidelines accept validation on the basis of criterion-validity, content-validity or construct-validity. As set forth above, with respect to the corporal examination, I am not satisfied that the technical infeasibility of criterion-validity studies has been established, although I have no doubt that the performance variable would be difficult to establish. But I find it difficult to justify imposing upon these defendants more stringent requirements than are mandated for federal executive agencies, and I have therefore concluded that the content-validity of the corporal examination, minimally established on this record, must be deemed acceptable.
With respect to all of the tests, then, we have (1) rather low correlations, (2) very low practical significance, (3) very high risks associated with promoting unqualified persons, and (4) substantial risks associated with wrongful rejection of qualified applicants.
It is appropriate, also, to consider carefully the consequences of rejecting the tests as a selection mechanism. There are undeniable institutional advantages in preservation of a "merit selection" system; indeed, it is required by law in this instance. Of course, Civil Service regulations cannot be permitted to preserve a selection process which violates Title VII. But in interpreting and applying Title VII and the pertinent guidelines, doubtful issues should be resolved in favor of preservation of the traditional ranking approach contemplated by Civil Service requirements. Moreover, it appears probable that, in the long run, the functioning of the Police Department and the morale of its personnel would be adversely affected if selection of persons to be promoted to supervisory positions were to be based, even partially, upon random selection. And it is at least arguable that if seniority in rank were to play a decisive role in selection from among a pool of qualified applicants, younger officers might be reluctant to apply for promotion, and there would be less incentive among the applicants to prepare themselves for the promotional examination. In short, there are decided institutional advantages in the competitive process itself.
All of the foregoing discussion assumes, of course, that racial bias plays no role in the design and implementation of the promotional examinations. There is no assertion that conscious racial prejudice is involved at any stage. There is, however, room for the argument that, when performance criteria are defined in terms of the perceptions of white incumbent high-level supervisors, the standards which emerge may be unwittingly slanted. For example, in connection with the third performance criterion used in the Haney sergeant study, "interaction with the public," the criterion was defined as follows:
"Please rank your sergeants on their ability to effectively interact with the public. Examples of this are handling complaints and keeping merchants happy. It is an ongoing behavior to develop and maintain good, effective relations with the public he serves."
The question of whether this truly represents a complete and accurate description of good "interaction with the public" is fairly debatable (the correctness of this Court's findings in Goode (COPPAR) v. Rizzo, 357 F. Supp. 1289 (1973), Aff'd 506 F.2d 542 (3d Cir. 1974), Rev'd, sub nom. Rizzo v. Goode, 423 U.S. 362, 96 S. Ct. 598, 46 L. Ed. 2d 561 (1976) was not challenged on appeal). But for present purposes, the point to be made is that "keeping merchants happy" might be understood as importing the racial attitudes of that segment of the community. A white rater might subconsciously assume that a white sergeant would be better able to meet that criterion.
It does appear, however, that there was no disparity in any of the ratings, as between white and black incumbents, and it is therefore reasonable to conclude that even subconscious racial bias did not taint the rating process.
It is beyond the province of a federal district court to impose its own perceptions of what constitutes good performance as a police officer, corporal, sergeant or detective, or any other rank. It is equally beyond the province of a federal district court to control management decisions concerning the acceptability of risks involved in alternative methods of selection for promotion within the Police Department, so long as such decisions do not serve to mask racial discrimination. If I were free to do so, I would agree with the plaintiffs that strict compliance with the EEOC guidelines ought to be required of any governmental agency before it is permitted to utilize selection procedures which have discriminatory impact. But that battle has, at least temporarily, been lost at the appellate level. See, E.g., Washington v. Davis, supra.
I am likewise persuaded that this Court's power to require the defendants to improve the quality of their testing procedures and analyses of their testing procedures is limited to matters affecting racially discriminatory impact.
Balancing all of the foregoing considerations, I have concluded that, insofar as plaintiffs seek to restrain the defendants from effectuating promotions from eligibility lists established pursuant to examinations heretofore administered, the Complaint must be dismissed. But in view of the rather obvious fact that these tests do serve to disqualify blacks at a disproportionately high rate compared with whites, and in view of the closeness of the question as to whether the minimal showing of job-relatedness has been made in this case (I. e., the sharp conflict among experts as to whether the guidelines have been complied with, and the distinct possibility that this Court's assessment of the evidence may later prove to have been erroneous), I have concluded that certain minimal steps should be required of the defendants with respect to future tests and future eligibility lists.
As set forth in the Findings of Fact, no differential item analysis has ever been performed with respect to any of the written examinations, for the purpose of determining whether particular questions may be the cause of a substantial portion of the racial differences in test scores. That omission, if uncorrected, might very well tip the scale in favor of a finding of discriminatory policy. It must be emphasized that this Court makes no such finding at this time, nor have the plaintiffs urged such a finding. But if discriminatory impact can be reduced, by minor clarifications or modifications of particular questions or phraseology, without any material decrease in the utility of the test as a screening device and at little or no cost, it is reasonable to conclude that the defendants should be perfectly willing to undertake such differential analyses. Performance of differential item analyses on all of the tests here in question, and upon any similar tests which may be used in the future for the purpose of determining eligibility for promotion, would thus serve two significant purposes: It would be useful as a check upon the validity of this Court's validity and utility findings (that is, it would either reinforce the validity of the tests, or demonstrate what changes would make them have less adverse racial impact), and it would also help further to rule out any successful charge of an unlawfully discriminatory policy.
AND NOW, this 23rd day of January, 1979, it is ORDERED that:
1. Except as set forth in this Order, insofar as plaintiffs seek to enjoin promotions within the Philadelphia Police Department on the basis of existing eligibility lists established through written examination, the Complaint is DISMISSED.
2. As a condition of the continued use of existing written examinations, or written examinations similar thereto for the establishment of future eligibility lists for such promotions, the defendants shall perform, or cause to be performed, by competent personnel, racial differential item analyses of such examinations, and shall report the results thereof to plaintiffs' counsel and to this Court. The Court reserves jurisdiction to consider and determine such other or further relief as the parties may contend would be appropriate in light of said differential item analyses.
3. Nothing in this Order is intended to preclude the defendants from taking steps to improve the quality, validity, and utility of any such examination, either through the use of their own personnel, or through consultation with outside experts.