Dossier n. 170/2008 [Abstract] Toward a national Observatory on the quality of continuing medical education

With the Agreement on continuing education of August 1st, 2007, the Italian Ministry of Health and Regions committed in developing a new program for continuing medical education (CME). A national Commission is entrusted with its governance. A national Observatory on the quality of continuing medical education, connected in a network with regional Observatories, will also be established.

The implementation of a national system to assess and promote the quality of continuing education through a network of independent observers working with shared criteria and methods has been proposed since some years. In particular in 2004 a pilot project was launched to outline the proposal for a national Observatory, its tools, a guideline for observations and a training path model for observers.

The pilot project was realized in 2005-2007 with the financing by the Ministry of Health and the participation of Emilia-Romagna Region (project and experimentation coordinator), Lazio, Lombardy, Marche, Puglia, Tuscany, Trentino, Valle d’Aosta and Veneto. The final results of the project were presented during a meeting in Bologna, May 21-22, 2007.

The volume collects some aspects of the project that are relevant for the new CME program, in particular to outline the role and tools of the regional Observatories.

The Italian program for CME involves all health professionals working for the National Health Service. It therefore needs to maintain its authority and usefulness and to avoid some actual risks of bureaucratization and commercialization. The proposed evaluative model considers the complex and diversified context, the necessary coherence with the CME program and the operational difficulties, and is meant to be both a verification and a promotion instrument.

The observation system is based on six elements:

• analysis of the documentation;
• observational visits during educational events;
• standardized observational criteria, tools and methods;
• independent observers with shared observational criteria and methods;
• qualitative and quantitative surveys on the opinion of events’ organizers and participants;
• common methods to elaborate observation results and to describe synthetic profiles.

The procedures to harmonized observers’ behaviours during visits and elaborations were studied, in particular concerning four aspects:

• evidence of coherence between the accredited project and its actual realization;
• quality of the class educational process as indicator of didactic skills;
• presence of conflict of interests;
• provider’s quality, evaluated through the analysis of the different evaluation instruments used during the event, as a partial but relevant indicator of the capacity to improve oneself from experience.

Evaluation reliability is a fundamental aspect of the proposal; some schemes for the observation of the first three elements (coherence, process quality and sponsoring) were therefore elaborated; they present also criteria and measuring scales. The fourth dimension (use of the evaluation) refers to past events; it can be studied during the observational visit only if it possible to consider past situations and if the reference person in the meeting is not the simple technical organizer of the event.

The proposed observational tools can also be used by education organizers in self-evaluation processes. This aspect will be fundamental when providers - and no longer events - will be accredited.

An education path model for observers in 4 modules with e-learning support is also presented in the volume. The first two modules are residential and introduce to CME principles and to the use observation and evaluation tools. A phase of field training with visits to events follows. The final module concerns visits evaluation and improving tools and procedures.

The proposed observation system cannot fully cover all the CME events that are normally realized. It aims instead to select some meaningful situations according to definite criteria (geographical distribution, type of provider, sponsoring forms, event features, etc.). Attention is focused on all providers, public and private, in particular on those that present more critical aspects.

To test other evaluation tools on quality, two national surveys were realized in October and November 2006 to study health professionals’ opinions on the education they have received since the beginning of the CME program and its future. For more details, Dossier n. 169 presents the main results of these studies.