Clinical trial monitoring – the present and the future

March 11, 2015 | By Márcio Barra

 

pharmu2

(Note: all references used are in the comments)

The modern clinical trial is a significant undertaking, and one that requires a multidisciplinary team(1). The current research team includes the principal investigator, sub – investigators, data managers, statisticians, clinical research coordinators (CRC), and the monitor or clinical research associate (CRA).

The International Conference on Harmonization (ICH) E6 Good Clinical Practices guidelines defines monitoring as “The act of overseeing the progress of a clinical trial, and of ensuring that it is conducted, recorded, and reported in accordance with the protocol, Standard Operating Procedures (SOPs), Good Clinical Practice (GCP), and the applicable regulatory requirement(s)”. CRAs are responsible for monitoring the trial, a task which ensures the integrity of the data that is being collected. It also allows the sponsor to closely follow the study centres, evaluate their conduct, and identify bottlenecks in patient recruitment(3).

In the past, it was the sponsor that would send their personnel to research sites to carry out the monitoring activities. This changed in the early nineties, with the emergence of contract research organizations (CROs). Like many other aspects of trial conduction, monitoring began to be outsourced (4).

The ICH guidelines state that data should be accurate, complete, legible, and timely. However, they do not provide instructions on how to approach data monitoring or individualized approaches for different types of trials. This leaves room for the industry to experiment with different strategies for trial monitoring, and with the advent of EDC, news ways to monitor clinical trial data are becoming more and more common.

On-Site Monitoring

On-site monitoring is the current industry standard, and one that has stuck with clinical trials for some decades (6). Here, the CRA is assigned to monitor a clinical trial or an observational study in a study centre or group of study centres, and then carries an in-person site evaluation (6). The most significant activities carried out by a CRA include: (1) the study initiation visit, where the CRA visits the centre to prepare the study staff for conducting the study; (2) several monitoring visits to a centre during the course of the trial; and (3) the study closure visit.

During monitoring visits, the CRA reviews the study documentation and source documents with the goal of identifying data entry errors and any missing data in the patient records or Case Report Forms (CRF). The CRA should also provide study documentation to the centre and evaluate the site’s staff familiarity and compliance with the protocol and study procedures. Throughout the entire process, the CRA essentially acts as an interface between the centre and the sponsor, giving the latter feedback on how the trial is being conducted.

According to a recent systematic literature review on on-site monitoring, such a strategy holds both benefits and disadvantages (39). Potential benefits of clinical trial monitoring include staff formation, better adhesion to the protocol and GCPs and improved communication between the staff.  Disadvantages include financial costs and time spent on-site visits. One important point raised by the review is that there was very little evidence found of the clinical and cost-effectiveness of on-site monitoring activities(39).

  • SDV

Source Data Verification (SDV), in which CRAs verify the data recorded in the trial’s CRFs against source documents, is the cornerstone of clinical trial on-site monitoring. CRAs spend a great deal of time going through both critical data (i.e data that, if inaccurate, would threaten the protection of the participants or the integrity of the study results) such as inclusion-exclusion criteria, efficacy and safety endpoint-related documentation, written informed consent, blinding procedures, drug accountability, as well as non-critical data points, which vary from study to study. Typically, most, if not all, data collected by a centre is verified by the CRA (100% or near 100% SDV ). The SDV process, depending on the complexity of the trial, can be a labour intensive process, especially when near 100% SDV is employed(3, 34).

The 100% or near 100% SDV stems from a notion shared by both sponsors and researchers that more is better, and on the safer side of regulations (34, 85). Some phase III and phase IV often carry enormous financial implications for the sponsors, and thus a high level of vigilance is presumed to help keep the bias contained and keep the results trustworthy(23).  However, There appears to be differing perceptions about the likelihood of substantial biases occurring without near 100% SDV, and on the potential magnitude of those biases affecting the perceived treatment effects and the study end results. The ICH E6 guideline does not state upper or lower limits for SDV, and many trials not sponsored by the pharmaceutical industry undergo limited, not 100% SDV monitoring, but are reported and interpreted in the literature as contributing equally reliable evidence of clinical effects(23).

  • Costs

On-site monitoring is an expensive activity, as CRAs have to travel to each research centre, pay for hotel expenses, as well as for the time spent on-site (34). On-site monitoring generally performed in industry trials add about 25–35% to the overall costs of a typical Phase 3 trial(30). Moreover, sponsors nowadays are faced with an increasingly complex research environment, including interactive voice response systems for randomization, imaging centres, central laboratories, CROs, regulatory pressure, and so forth, which in turns significantly increases the amount of collected data and, consequently, the monitoring workload (3). The costly nature of on-site monitoring has resulted in some studies becoming too expensive to perform, while others are slowed down and the overall progress of research is impaired(23). These financial and logistical hurdles to the design and conduct of affordable clinical trials are especially limiting for small, independent clinical trials(30).

Centralized Monitoring and EDC

Centralized Monitoring, or off-site monitoring, is where the CRA, or any assigned sponsor personnel, conduct a remote evaluation of the study centre. Centralized monitoring was made possible by the evolution of modern information technologies, and the advent of EDC, most noticeably the electronic case report form (eCRF). Historically, CRFs were originally paper-based. Whereas in the past, there was a considerable time lag between the data collection point, and the time when it was entered into the computer system in the sponsor’s facility(68), nowadays, EDC and the eCRFs brought a new dynamic to the clinical trial enterprise. Most modern clinical trials rely on EDC for their CRFs, with direct data entry being an option in many research centres. Some protocols even allow direct data entry by subjects(44).

Central monitoring has its own sets of strengths. Central monitoring technologies allow a greater degree of flexibility, additional statistical monitoring strategies, and allow for errors and discrepancies to be identified earlier since the data can be monitored in real-time by a data manager(88, 89). Additional central statistical monitoring techniques include checks for missing or invalid data, calendar checks, checks for unusual data patterns, assessment of rates of reporting, checks of performance indicators and comparisons with external sources(29). Central monitoring can, in some cases, detect data fabrication, as in a recent large multicenter trial, the data of 438 patients was fabricated at one site, and was only detected though analysis of statistical anomalies(29).

It should be pointed that central monitoring might not be adequate for all trials. If, for example, the burden of paperwork to be reviewed centrally is excessive, as may be the case for some large trials enrolling a considerable number of participants, central monitoring might not be appropriate. Central monitoring might be more suitable for large trials involving more sites with fewer participants per site, and for double-blind superiority trials using an objective endpoint such as mortality

While EDC holds considerable potential, both the methodologies and tools needed for it are very complex. Welker identified some of the barriers to the implementation of EDC systems in clinical research. These include user input, motivation and communication, regulatory requirements, timing of implementation, software installation, proper graphical user interface, identification of early technology adopters, patient participation, availability of technology and last, but not least, the costs associated with implementing and maintaining EDC technologies in all research centres for a trial. While implementation costs are high, substantial cost reductions have been demonstrated when compared to traditional paper based data capture methods (76).

On Site Monitoring versus Centralized Monitoring approaches and EDC

Through the years , several studies were published highlighting the benefits and potential limitations of centralized monitoring and EDC, and compared it to traditional paper based data collection forms.

The STARBRITE clinical study implemented, in parallel to the ongoing data collection procedures, a single source EDC system in a clinical trial. While small in scope and not comparing directly central monitoring approaches to on-site monitoring, the study gave some preliminary evidence that single source EDC can ease the burden of validating the source of clinical research data and improve quality and efficiency by eliminating manual and redundant data entry(47).

Walther et al (75) compared a series of EDC technologies to the traditional paper based data collection in regards to duration of data capture and accuracy. Results showed a considerable reduction in the time from data collection to database lock, and the study concluded that EDC can be more time effective than the standard, paper based data capturing process followed by double entry and verification. However, the successful implementation of EDC was shown to require adjustment of work processes and reallocation of resources. Whether EDC is more accurate than the paper-based method could not be confirmed in this pilot study.

Weiler et al. (70) found no discernible difference in a paper based data capture instrument versus an EDC device in a randomized crossover trial of 87 adults with allergic rhinitis, and advised for a sensible, trial adjusted approach when selecting the tools for clinical trial data collection. In another randomised controlled trial, clinical research coordinators were asked to collect patient information on a hand held computer in parallel with a paper based CRF. High error rates occurred with hand held computers, compared with paper based CRFs – 67.5 error per 1000 fields, against the accepted error rate of 10 per 10,000 field for paper-based CRF data entry – highlighting the need for proper staff training in EDC before it can be used in clinical research(71).

The European Childhood Obesity Project implemented EDC for their data collection procedures, with notebooks being used for 73.0% of the visits. EDC was shown to reduce significantly the need for after-trial data checks, but the planning and implementation processes of the technologies was deemed more time consuming(60). The most recent study evaluated pre-visit remote SDV compared to traditional on-site SDV in five hospitals from two NIH sponsored networks, ARDS and ChiLDREN. The results showed that 99.5 % of the ARDS network data values were found remotely, and 100% of the ChiLDREN network data values were verified remotely, demonstrating that central monitoring is a viable option for SDV(90).

Bakobaki et al.(85) were the first to directly compare on-site monitoring to central monitoring. They evaluated the extent to which findings identified during half of a phase III trial could have been identified through central monitoring. The results showed that over 90% of the findings identified from the review of site monitoring reports, which amounted to 31 person days to conduct the four site visits, could have been found by central monitoring. Of the 5% that would be unlikely to be identified centrally, there were only two major findings, which were unlikely to have a direct impact on the results.

Eisenstein et al.(6) argued, by simulating the costs of a hypothetical mega-trial with an assumed number of 24 monitoring visits per site, with a $10,000 per patient site payment, that source document verification should be centralized, where appropriate. By using minimal on-site monitoring at the research site to a limited, previous selected, set of records and to ensure some form of personal contact with the study centre, the simulation showed savings of clinical trial costs of more than 20%.

Ongoing efforts

One of the biggest ongoing projects that seeks to understand whether a reduced on-site monitoring strategy is equivalent to extensive full, on-site intensive monitoring regarding the occurrence of serious or critical flaws is the ADAMON project (ADAMON, for ADAptiertes MONitoring)(91). This initiative, which began in 2008 and is funded by the German Ministry of Education and Research, includes twelve different clinical trials, including phase II, III, and IV and non-commercial clinical trials, with a total of 3200 patients. Each trial is randomized to one of two possible monitoring approaches: either a trial specific, centralized monitoring approach with minimum on-site monitoring or a traditional, intensive on-site monitoring approach. The main outcome of the ADAMON project is occurrence of serious or critical findings concerning patient safety and data validity. The project is funded until December 2014, and results are expected in 2014. A similar, completed, project is the French OPTIMON (OPTIMON, for OPTImisation of MONitoring), which will compare these two monitoring approaches. The results are still to be published.

In 2015, the UK’s Medical Research Council’s Clinical Trials Unit is expected to release the results of its TEMPER study (Targeted Monitoring, Prospective Evaluation and Refinement). In this trial, a triggered on-site monitoring strategy for a multi-centre cancer clinical trial will be compared to a non-triggered, non-risk sensitive approach.

The CTTI, and the FDA’s and EMA’s guidance documents

In 2007, the United States Food and Drug Administration (FDA) and Duke University created a public-private partnership entitled Clinical Trials Transformation Initiative (CTTI), with the purpose of identifying practices to help address two questions: Do clinical trials have to be so heavy on time and resources expenditures, and what can be done so that randomized comparison can answer more clinical questions?

The CTTI made monitoring the focus of its first project, with the goal of helping identify the best practices to help sponsors select the most appropriate monitoring methods for a trial, through an evidence based methodology. The preliminary recommendations of CTTI placed emphasis on the creation of an efficient monitoring plan alongside the protocol, training of individuals regarding data collection, the use of risk based approaches and the allocation of resources to more critical data points in lieu of less critical ones, while encouraging sponsors to discuss their monitoring plan with the FDA(92).

Following the inception of the CTTI, the FDA released in 2011 a draft guidance entitled “Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring”, with the final version being released in 2013(93). The guidance advocates the use of risk-based monitoring approaches, driven by centralized and off-site monitoring methods, and the use of the modern EDC systems to collect and analyse data, with less emphasis on on-site monitoring. In 2013, The European Medicines Agency (EMA) also released a draft for consultation entitled “Reflection paper on risk based quality management in clinical trials”(94).

Both the FDA and EMA, in their releases, acknowledge that traditional 100 % SDV-based monitoring approaches are not always the most efficient, and are not always warranted. There is evidence that errors in non-critical data points, like ones related to concomitant medications, demographic data, or any other collected information that does not concern study endpoints ,do not affect a clinical trial outcome(29, 85), and thus might not need such a tight monitoring oversight. Both documents show the agencies encouraging sponsors to conduct a risk-assessment evaluation of their clinical trials.  This is to identify events that could affect the quality of the clinical trial data or the performance of clinical trial processes, determine the probability of such events occurring, their potential impact on human safety and trial integrity, and the extent to which these risks can be detected.  Following this evaluation, sponsors are advised to plan out a monitoring strategy proportional to the risk and focused on critical data.

A key point seems to be finding the balance between central and on-site monitoring. The existence of an experienced team who can define the important checks at the time of CRF design and that are aware of the risks is paramount when designing the monitoring plan (3, 41, 95). This should help guarantee that a proper number of on-site monitoring visits is scheduled and that a robust central monitoring approach is in place, all adjusted to risk.

Authors who have previously advocated the implementation of risk-based monitoring and risk adapted monitoring on clinical trials include Brosteanu et al.(41). Through a survey of existing monitoring practices and expert’s opinions, they developed a risk assessment score and analysis procedure for use in clinical trials.

CONCLUSION

Clinical trial monitoring seems to be standing at a crossroad. On one hand, sponsors and regulatory agencies are pressing for a more focused, risk adapted monitoring approach to clinical trials, while on the other, there are considerable bottlenecks in order for these strategies to be used to their full potential. Not all clinical trial centres are ready to implement the needed EDC technologies, staff training is still an issue, and some clinical trials would benefit more from these approaches than others. But the fact remains that, as costs keep rising and cost containment measures are employed by clinical trial sponsors, it comes as no surprise that alternatives to the costly on-site monitoring are becoming more and more attractive to stakeholders.

1 comment
  1. 1. Rico-Villademoros F, Hernando T, Sanz JL, Lopez-Alonso A, Salamanca O, Camps C, et al. The role of the clinical research coordinator–data manager–in oncology clinical trials. BMC Medical Research Methodology. 2004;4:6. PubMed PMID: 15043760. Pubmed Central PMCID: PMC406503.
    2. Center for Drug E, Research CfBE, Research ICoH. Guidance for industry E6 good clinical practice, consolidated guidance Rockville, MD: U.S. Dept. of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Control : Center for Biologics Evaluation and Research; 2000. Available from: http://purl.access.gpo.gov/GPO/LPS113971.
    3. Williams GW. The other side of clinical trial monitoring; assuring data quality and procedural adherence. Clinical Trials. 2006;3(6):530-7. PubMed PMID: 17170037.
    4. Beach JE. Clinical trials integrity: a CRO perspective. Accountability in Research. 2001;8(3):245-60. PubMed PMID: 12481762. Pubmed Central PMCID: Source: KIE. 104936.
    5. Eisenstein EL, Lemons PW, 2nd, Tardiff BE, Schulman KA, Jolly MK, Califf RM. Reducing the costs of phase III cardiovascular clinical trials. American Heart Journal. 2005;149(3):482-8. PubMed PMID: 15864237.
    6. Eisenstein EL, Collins R, Cracknell BS, Podesta O, Reid ED, Sandercock P, et al. Sensible approaches for reducing clinical trial costs. Clinical Trials. 2008;5(1):75-84. PubMed PMID: 18283084.
    7. Pronker E, Geerts BF, Cohen A, Pieterse H. Improving the quality of drug research or simply increasing its cost? An evidence-based study of the cost for data monitoring in clinical trials. British Journal of Clinical Pharmacology. 2011;71(3):467-70. PubMed PMID: 21284707. Pubmed Central PMCID: PMC3045557.
    8. O’Leary E, Seow H, Julian J, Levine M, Pond GR. Data collection in cancer clinical trials: Too much of a good thing? Clinical Trials. 2013;10(4):624-32. PubMed PMID: 23785066.
    9. Uren SC, Kirkman MB, Dalton BS, Zalcberg JR. Reducing clinical trial monitoring resource allocation and costs through remote access to electronic medical records. Journal of oncology practice/American Society of Clinical Oncology. 2013;9(1):e13-6. PubMed PMID: 23633977. Pubmed Central PMCID: PMC3545670.
    10. McGillick KJ, Fernandes R. The role of the nurse clinical research associate in testing a new drug. Nursing Forum. 1980;19(4):379-84. PubMed PMID: 7031617.
    11. Larkin ME, Lorenzi GM, Bayless M, Cleary PA, Barnie A, Golden E, et al. Evolution of the study coordinator role: the 28-year experience in Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC). Clinical Trials. 2012;9(4):418-25. PubMed PMID: 22729476. Pubmed Central PMCID: NIHMS508533
    PMC3815574.
    12. Hurst C, Dennis BP. Developing a clinical research associate training program at Dillard University: the impact of collaboration. ABNF Journal. 2013;24(4):104-10. PubMed PMID: 24303584.
    13. Glaser RD, Wallace C, Bass SW. Monitoring and recording clinical trials. Medical Instrumentation. 1979;13(2):109-12. PubMed PMID: 431427.
    14. Stowe SM, Hammond GD, Chard R, Hornberger JA, Honour RC, Sposto R, et al. Monitoring clinical research. A report from the Childrens Cancer Study Group. American Journal of Clinical Oncology. 1984;7(5):557-66. PubMed PMID: 6507379.
    15. Morse MA, Califf RM, Sugarman J. Monitoring and ensuring safety during clinical research. JAMA. 2001;285(9):1201-5. PubMed PMID: 11231751.
    16. Parry D. Monitoring clinical trials. Several points are contentious. BMJ. 2001;323(7326):1425. PubMed PMID: 11778590. Pubmed Central PMCID: Source: KIE. 103084.
    17. Takayanagi R, Watanabe K, Nakahara A, Nakamura H, Yamada Y, Suzuki H, et al. Items of concern associated with source document verification of clinical trials for new drugs. Yakugaku Zasshi – Journal of the Pharmaceutical Society of Japan. 2004;124(2):89-92. PubMed PMID: 14978950.
    18. Miller WR, Moyers TB, Arciniega L, Ernst D, Forcehimes A. Training, supervision and quality monitoring of the COMBINE Study behavioral interventions. Journal of Studies on Alcohol – Supplement. 2005 (15):188-95; discussion 68-9. PubMed PMID: 16223070.
    19. Kirwan BA, Lubsen J, de Brouwer S, van Dalen FJ, Pocock SJ, Clayton T, et al. Quality management of a large randomized double-blind multi-centre trial: the ACTION experience. Contemporary Clinical Trials. 2008;29(2):259-69. PubMed PMID: 18029294.
    20. Sharples K, Fleming TR, MacMahon S, Moore A, Reid I, Scoggins B. Monitoring clinical trials. New Zealand Medical Journal. 1998;111(1072):322-5. PubMed PMID: 9765633.
    21. Marschner N. [Clinical investigations: innovative management of data. Electronic Data Capture: IOStudy Office]. Zentralblatt fur Gynakologie. 2001;123(8):441-3. PubMed PMID: 11562806.
    22. Crerand WJ, Lamb J, Rulon V, Karal B, Mardekian J. Building data quality into clinical trials. Journal of Ahima. 2002;73(10):44-6, 8-53, 2; quiz 5-6. PubMed PMID: 12432815.
    23. Lewis JA. The European regulatory experience. Statistics in Medicine. 2002;21(19):2931-8. PubMed PMID: 12325109.
    24. O’Neill RT. Regulatory perspectives on data monitoring. Statistics in Medicine. 2002;21(19):2831-42. PubMed PMID: 12325099.
    25. Sahoo U, Bhatt A. Electronic data capture (EDC)–a new mantra for clinical trials. Quality Assurance. 2003;10(3-4):117-21. PubMed PMID: 15764550.
    26. D’Agostino RB, Sr., Massaro JM. New developments in medical clinical trials. Journal of Dental Research. 2004;83 Spec No C:C18-24. PubMed PMID: 15286116.
    27. Velazquez I, Navarro X, Cobos A. [Electronic data capture. Impact on the quality of the clinical research]. Medicina Clinica. 2004;122 Suppl 1:11-5. PubMed PMID: 14980154.
    28. Lopez-Carrero C, Arriaza E, Bolanos E, Ciudad A, Municio M, Ramos J, et al. Internet in clinical research based on a pilot experience. Contemporary Clinical Trials. 2005;26(2):234-43. PubMed PMID: 15837443.
    29. Baigent C, Harrell FE, Buyse M, Emberson JR, Altman DG. Ensuring trial validity by data quality assurance and diversification of monitoring methods. Clinical Trials. 2008;5(1):49-55. PubMed PMID: 18283080.
    30. Duley L, Antman K, Arena J, Avezum A, Blumenthal M, Bosch J, et al. Specific barriers to the conduct of randomized trials. Clinical Trials. 2008;5(1):40-8. PubMed PMID: 18283079.
    31. Howells K. Living without a data management system. Idrugs. 2008;11(10):733-7. PubMed PMID: 18828073.
    32. Davis B. Changing regulation of clinical trials in Europe. Expert Review of Clinical Pharmacology. 2010;3(4):453-8. PubMed PMID: 22111676.
    33. Shah J, Rajgor D, Pradhan S, McCready M, Zaveri A, Pietrobon R. Electronic data capture for registries and clinical trials in orthopaedic surgery: open source versus commercial systems. Clinical Orthopaedics & Related Research. 2010;468(10):2664-71. PubMed PMID: 20635174. Pubmed Central PMCID: PMC3049639.
    34. De S. Hybrid approaches to clinical trial monitoring: Practical alternatives to 100% source data verification. Perspectives in Clinical Research. 2011;2(3):100-4. PubMed PMID: 21897885. Pubmed Central PMCID: PMC3159208.
    35. Morrison BW, Cochran CJ, White JG, Harley J, Kleppinger CF, Liu A, et al. Monitoring the quality of conduct of clinical trials: a survey of current practices. Clinical Trials. 2011;8(3):342-9. PubMed PMID: 21730082.
    36. Bakobaki J, Joffe N, Burdett S, Tierney J, Meredith S, Stenning S. A systematic search for reports of site monitoring technique comparisons in clinical trials. Clinical Trials. 2012;9(6):777-80. PubMed PMID: 23059772.
    37. Cornu C, Binquet C, Thalamas C, Vigouroux C, Gaillard S, Ginhoux T, et al. [Public clinical trials: which kind of monitoring should be used?]. Therapie. 2013;68(3):135-41. PubMed PMID: 23886457.
    38. Eapen ZJ, Vavalle JP, Granger CB, Harrington RA, Peterson ED, Califf RM. Rescuing clinical trials in the United States and beyond: a call for action. American Heart Journal. 2013;165(6):837-47. PubMed PMID: 23708153.
    39. Macefield RC, Beswick AD, Blazeby JM, Lane JA. A systematic review of on-site monitoring methods for health-care randomised controlled trials. Clinical Trials. 2013;10(1):104-24. PubMed PMID: 23345308.
    40. Journot V, Pignon JP, Gaultier C, Daurat V, Bouxin-Metro A, Giraudeau B, et al. Validation of a risk-assessment scale and a risk-adapted monitoring plan for academic clinical research studies–the Pre-Optimon study. Contemporary Clinical Trials. 2011;32(1):16-24. PubMed PMID: 20951234.
    41. Brosteanu O, Houben P, Ihrig K, Ohmann C, Paulus U, Pfistner B, et al. Risk analysis and risk adapted on-site monitoring in noncommercial clinical trials. Clinical Trials. 2009;6(6):585-96. PubMed PMID: 19897532.
    42. Tudur Smith C, Stocken DD, Dunn J, Cox T, Ghaneh P, Cunningham D, et al. The value of source data verification in a cancer clinical trial. PLoS ONE [Electronic Resource]. 2012;7(12):e51623. PubMed PMID: 23251597. Pubmed Central PMCID: PMC3520949.
    43. Dyck PJ, Turner DW, Davies JL, O’Brien PC, Dyck PJ, Rask CA, et al. Electronic case-report forms of symptoms and impairments of peripheral neuropathy. Canadian Journal of Neurological Sciences. 2002;29(3):258-66. PubMed PMID: 12195616.
    44. Brandt CA, Argraves S, Money R, Ananth G, Trocky NM, Nadkarni PM. Informatics tools to improve clinical research study implementation. Contemporary Clinical Trials. 2006;27(2):112-22. PubMed PMID: 16388990.
    45. Soran A, Nesbitt L, Mamounas EP, Lembersky B, Bryant J, Anderson S, et al. Centralized medical monitoring in phase III clinical trials: the National Surgical Adjuvant Breast and Bowel Project (NSABP) experience. Clinical Trials. 2006;3(5):478-85. PubMed PMID: 17060221.
    46. Huffstutter J, Craig WD, Schimizzi G, Harshbarger J, Lisse J, Kasle S, et al. A multicenter, randomized, open study to evaluate the impact of an electronic data capture system on the care of patients with rheumatoid arthritis. Current Medical Research & Opinion. 2007;23(8):1967-79. PubMed PMID: 17626700.
    47. Kush R, Alschuler L, Ruggeri R, Cassells S, Gupta N, Bain L, et al. Implementing Single Source: the STARBRITE proof-of-concept study. Journal of the American Medical Informatics Association. 2007;14(5):662-73. PubMed PMID: 17600107. Pubmed Central PMCID: PMC1975790.
    48. Estellat C, Tubach F, Costa Y, Hoffmann I, Mantz J, Ravaud P. Data capture by digital pen in clinical trials: a qualitative and quantitative study. Contemporary Clinical Trials. 2008;29(3):314-23. PubMed PMID: 17974503.
    49. Kaushik S, Khan A, Kaushik S, Lindenmayer JP. Interface for electronic data capture systems for clinical trials by optimal utilization of available hospital resources. AMIA 2008;Annual Symposium Proceedings/AMIA Symposium.:998. PubMed PMID: 18999083.
    50. Nahm ML, Pieper CF, Cunningham MM. Quantifying data quality for clinical trials using electronic data capture. PLoS ONE [Electronic Resource]. 2008;3(8):e3049. PubMed PMID: 18725958. Pubmed Central PMCID: PMC2516178.
    51. Bellamy N, Wilson C, Hendrikz J, Patel B, Dennison S. Electronic data capture (EDC) using cellular technology: implications for clinical trials and practice, and preliminary experience with the m-Womac Index in hip and knee OA patients. Inflammopharmacology. 2009;17(2):93-9. PubMed PMID: 19139830.
    52. El Emam K, Jonker E, Sampson M, Krleza-Jeric K, Neisa A. The use of electronic data capture tools in clinical trials: Web-survey of 259 Canadian trials. Journal of Medical Internet Research. 2009;11(1):e8. PubMed PMID: 19275984. Pubmed Central PMCID: PMC2762772.
    53. Ene-Iordache B, Carminati S, Antiga L, Rubis N, Ruggenenti P, Remuzzi G, et al. Developing regulatory-compliant electronic case report forms for clinical trials: experience with the demand trial. Journal of the American Medical Informatics Association. 2009;16(3):404-8. PubMed PMID: 19261946. Pubmed Central PMCID: PMC2732224.
    54. Ruping S, Wegener D, Sfakianakis S, Sengstag T. Workflows for intelligent monitoring using proxy services. Studies in Health Technology & Informatics. 2009;147:277-82. PubMed PMID: 19593067.
    55. Shapiro M, Silva SG, Compton S, Chrisman A, DeVeaugh-Geiss J, Breland-Noble A, et al. The child and adolescent psychiatry trials network (CAPTN): infrastructure development and lessons learned. Child & Adolescent Psychiatry & Mental Health [Electronic Resource]. 2009;3(1):12. PubMed PMID: 19320979. Pubmed Central PMCID: PMC2673205.
    56. Arab L, Hahn H, Henry J, Chacko S, Winter A, Cambou MC. Using the web for recruitment, screen, tracking, data management, and quality control in a dietary assessment clinical validation trial. Contemporary Clinical Trials. 2010;31(2):138-46. PubMed PMID: 19925884. Pubmed Central PMCID: NIHMS189311
    PMC2862235.
    57. Nahm M, Shepherd J, Buzenberg A, Rostami R, Corcoran A, McCall J, et al. Design and implementation of an institutional case report form library. Clinical Trials. 2011;8(1):94-102. PubMed PMID: 21163853. Pubmed Central PMCID: NIHMS320774
    PMC3494996.
    58. Fraccaro P, Giacomini M. A web-based tool for patients cohorts and Clinical Trials management. Studies in Health Technology & Informatics. 2012;180:554-8. PubMed PMID: 22874252.
    59. Murphy SN, Dubey A, Embi PJ, Harris PA, Richter BG, Turisco F, et al. Current state of information technologies for the clinical research enterprise across academic medical centers. Clinical and translational science. 2012;5(3):281-4. PubMed PMID: 22686207.
    60. Pawellek I, Richardsen T, Oberle D, Grote V, Koletzko B. Use of electronic data capture in a clinical trial on infant feeding. European Journal of Clinical Nutrition. 2012;66(12):1342-3. PubMed PMID: 23211655.
    61. Yamamoto K, Yamanaka K, Hatano E, Sumi E, Ishii T, Taura K, et al. An eClinical trial system for cancer that integrates with clinical pathways and electronic medical records. Clinical Trials. 2012;9(4):408-17. PubMed PMID: 22605791.
    62. Clarke DE, Narrow WE, Regier DA, Kuramoto SJ, Kupfer DJ, Kuhl EA, et al. DSM-5 field trials in the United States and Canada, Part I: study design, sampling strategy, implementation, and analytic approaches. American Journal of Psychiatry. 2013;170(1):43-58. PubMed PMID: 23111546.
    63. Fraccaro P, Dentone C, Fenoglio D, Giacomini M. Multicentre clinical trials’ data management: a hybrid solution to exploit the strengths of electronic data capture and electronic health records systems. Informatics for health & social care. 2013;38(4):313-29. PubMed PMID: 23957714.
    64. Jansen ME, Kollbaum PS, McKay FD, Rickert ME. Factors influencing the electronic capture of patient-reported contact lens performance data. Contact Lens & Anterior Eye. 2013;36(3):130-5. PubMed PMID: 23279731.
    65. Journot V, Perusat-Villetorte S, Bouyssou C, Couffin-Cadiergues S, Tall A, Fagard C, et al. Preserving participant anonymity during remote preenrollment consent form checking. Clinical Trials. 2013;10(3):460-2. PubMed PMID: 23559559.
    66. Pietanza MC, Basch EM, Lash A, Schwartz LH, Ginsberg MS, Zhao B, et al. Harnessing technology to improve clinical trials: study of real-time informatics to collect data, toxicities, image response assessments, and patient-reported outcomes in a phase II clinical trial. Journal of Clinical Oncology. 2013;31(16):2004-9. PubMed PMID: 23630218.
    67. Franklin JD, Guidry A, Brinkley JF. A partnership approach for Electronic Data Capture in small-scale clinical trials. Journal of Biomedical Informatics. 2011;44 Suppl 1:S103-8. PubMed PMID: 21651992. Pubmed Central PMCID: NIHMS305950
    PMC3202046.
    68. Kowitt L, Marlin RL. A comprehensive on-line collection system. Drug Information Journal. 1984;18(1):9-14. PubMed PMID: 10266402.
    69. Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. Journal of Asthma. 2003;40(7):751-62. PubMed PMID: 14626331.
    70. Weiler K, Christ AM, Woodworth GG, Weiler RL, Weiler JM. Quality of patient-reported outcome data captured using paper and interactive voice response diaries in an allergic rhinitis study: is electronic data capture really better? Annals of Allergy, Asthma, & Immunology. 2004;92(3):335-9. PubMed PMID: 15049397.
    71. Shelby-James TM, Abernethy AP, McAlindon A, Currow DC. Handheld computers for data entry: high tech has its problems too. Trials [Electronic Resource]. 2007;8:5. PubMed PMID: 17309807. Pubmed Central PMCID: PMC1804282.
    72. Richter JG, Becker A, Koch T, Nixdorf M, Willers R, Monser R, et al. Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires. Annals of the Rheumatic Diseases. 2008;67(12):1739-41. PubMed PMID: 18647853.
    73. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value in Health. 2009;12(4):419-29. PubMed PMID: 19900250.
    74. O’Halloran JP, Kemp AS, Salmon DP, Tariot PN, Schneider LS. Psychometric comparison of standard and computerized administration of the Alzheimer’s Disease Assessment Scale: Cognitive Subscale (ADASCog). Current Alzheimer Research. 2011;8(3):323-8. PubMed PMID: 21314622.
    75. Walther B, Hossin S, Townend J, Abernethy N, Parker D, Jeffries D. Comparison of electronic data capture (EDC) with the standard data capture method for clinical trial data. PLoS ONE [Electronic Resource]. 2011;6(9):e25348. PubMed PMID: 21966505. Pubmed Central PMCID: PMC3179496.
    76. Welker JA. Implementation of electronic data capture systems: barriers and solutions. Contemporary Clinical Trials. 2007;28(3):329-36. PubMed PMID: 17287151.
    77. Atreja A, Achkar JP, Jain AK, Harris CM, Lashner BA. Using technology to promote gastrointestinal outcomes research: a case for electronic health records. American Journal of Gastroenterology. 2008;103(9):2171-8. PubMed PMID: 18844611.
    78. El Fadly A, Lucas N, Rance B, Verplancke P, Lastic PY, Daniel C. The REUSE project: EHR as single datasource for biomedical research. Studies in Health Technology & Informatics. 2010;160(Pt 2):1324-8. PubMed PMID: 20841899.
    79. Kohl CD, Garde S, Knaup P. Facilitating secondary use of medical data by using openEHR archetypes. Studies in Health Technology & Informatics. 2010;160(Pt 2):1117-21. PubMed PMID: 20841857.
    80. Goodman K, Krueger J, Crowley J. The automatic clinical trial: leveraging the electronic medical record in multisite cancer clinical trials. Current Oncology Reports. 2012;14(6):502-8. PubMed PMID: 22907283. Pubmed Central PMCID: NIHMS402213 [Available on 12/01/13]
    PMC3490046 [Available on 12/01/13].
    81. Xu W, Guan Z, Sun J, Wang Z, Geng Y. Development of an open metadata schema for prospective clinical research (openPCR) in China. Methods of Information in Medicine. 2014;53(1):39-46. PubMed PMID: 24317371.
    82. Venet D, Doffagne E, Burzykowski T, Beckers F, Tellier Y, Genevois-Marlin E, et al. A statistical approach to central monitoring of data quality in clinical trials. Clinical Trials. 2012;9(6):705-13. PubMed PMID: 22684241.
    83. Kirkwood AA, Cox T, Hackshaw A. Application of methods for central statistical monitoring in clinical trials. Clinical Trials. 2013;10(5):783-806. PubMed PMID: 24130202.
    84. Pogue JM, Devereaux PJ, Thorlund K, Yusuf S. Central statistical monitoring: detecting fraud in clinical trials. Clinical Trials. 2013;10(2):225-35. PubMed PMID: 23283577.
    85. Bakobaki JM, Rauchenberger M, Joffe N, McCormack S, Stenning S, Meredith S. The potential for central monitoring techniques to replace on-site monitoring: findings from an international multi-centre clinical trial. Clinical Trials. 2012;9(2):257-64. PubMed PMID: 22064687.
    86. Mealer M, Kittelson J, Thompson BT, Wheeler AP, Magee JC, Sokol RJ, et al. Remote source document verification in two national clinical trials networks: a pilot study. PLoS ONE [Electronic Resource]. 2013;8(12):e81890. PubMed PMID: 24349149. Pubmed Central PMCID: PMC3857788.
    87. Walden A, Nahm M, Barnett ME, Conde JG, Dent A, Fadiel A, et al. Economic analysis of centralized vs. decentralized electronic data capture in multi-center clinical studies. Studies in Health Technology & Informatics. 2011;164:82-8. PubMed PMID: 21335692. Pubmed Central PMCID: NIHMS320780
    PMC3506018.
    88. Marks RG, Conlon M, Ruberg SJ. Paradigm shifts in clinical trials enabled by information technology. Stat Med. 2001 Sep 15-30;20(17-18):2683-96. PubMed PMID: 11523076. Epub 2001/08/28. eng.
    89. Stafford PB, Garrett A. Using Real-time Data to Drive Better Decisions, Faster. Drug Information Journal. 2011 July 1, 2011;45(4):495-502.
    90. Mealer M, Kittelson J, Thompson BT, Wheeler AP, Magee JC, Sokol RJ, et al. Remote Source Document Verification in Two National Clinical Trials Networks: A Pilot Study. PloS one. 2013;8(12):e81890.
    91. Brosteanu O, Houben P, Ihrig K, Ohmann C, Paulus U, Pfistner B, et al. Risk analysis and risk adapted on-site monitoring in noncommercial clinical trials. Clinical Trials. 2009 December 1, 2009;6(6):585-96.
    92. Grignolo A. The Clinical Trials Transformation Initiative (CTTI). Annali dell’Istituto superiore di sanita. 2011;47(1):14-8. PubMed PMID: 21430332. Epub 2011/03/25. eng.
    93. Administration USDoHaHS-FaD. Guidance for Industry Oversight of Clinical Investigations — A Risk-Based Approach to Monitoring. 2013.
    94. Group EMA-CTF. Reflection paper on risk based quality management in clinical trials 2011 4 August 2011.
    95. Knatterud GL, Rockhold FW, George SL, Barton FB, Davis CE, Fairweather WR, et al. Guidelines for quality assurance in multicenter trials: a position paper. Control Clin Trials. 1998 Oct;19(5):477-93. PubMed PMID: 9741868. Epub 1998/09/19. eng.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: