Evaluating effectiveness of the trainingof civil servants
Level 1: Reaction
•The degree to which participants find the training favorable, engaging and relevant to their jobs
Kirkpatrick model after development of the program complicates training value assessment therefore it is necessary to consider all four levels of development, productivities and estimates of the program.
Return of expectations (ROE): as a rule, when new programs, the organizations offering training are necessary begin to develop samples of the appropriate new programs. According to Kirkpatrick, it is much more effective to estimate requirements, without disturbing new ways and also to predict results of their training at a basis of dialogue with interested parties. Subsequently these results will be presented at the fourth level. Researches have shown that 90 % of educational resources are used for development, prepara- tion and holding training events, and only 15 % work at a workplace [7]. Actions which follow training pro- cess are crucial for the high level of teaching.
The template includes tools for measuring each level: at the first level they are evaluated (questionnaires, opinion polls), interviews, focus groups; Tests on the second level of prepared materials, inspections, action plans, training others (coaching, coaching, etc.); Behavioral sheets at the third level, behavior overview, workplace employee surveys, post-program reviews, post-training surveys, performance monitoring, focus groups and fourth-level indicators, and the achievement of the necessary individual and organizational results. Short-term observations and criteria that determine the directional orientation of managers , the level of satisfaction of managers, the level of customer satisfaction and the satisfaction of other employees. Round, frame shift, as a means of rational use of time.
As an example, applying additional training programs to public servants in Academies of public admin- istration, some elements of assessment model offered here now are used rationally. The cycle of training is characterized by 4 levels: definition of needs for training, development of the training program; ensuring training; learning efficiency assessment.
The Academy uses three levels of performance evaluation: trainee training - where initial control and final control, design work is used; the opinions of the participants are established in the form of a communication, quantitative estimates are established; Improvement of the curriculum. Teachers who develop the procedure for approving the curriculum are guided by the results of the programs on the basis of their feedback and feedback from participants, choosing optional questionnaires during post-monitoring with the Academy to determine the level of knowledge acquired during the academic period. In addition, the Academy is submitted every two years through surveys of state bodies. Then determine the impact of training activities. Graduates of the Academy are divided into three groups: participants of retraining courses, participants of seminars and masters of professional growth, doctors. The results of the survey showed the level of customer satisfaction with the curriculum. At the same time, the author of the state bodies of the Republic of Kazakhstan «Center for assessing the effectiveness of retraining and refresher courses for civil servants, civil servants do not meet the needs and goals of the government» between the Academy and the results of the assessment of the center due to the conclusions of the contradiction, only 10 % of The total number of studies is covered by the Academy, and many tools and methods of evaluation [8] are explained.
The Academic system of assessment of results of training has been at first sight created, and her realiza- tion is organized properly. However it is possible to see that at all stages of educational process and at all evaluation stages public authorities don't play a key role as clients, and the main, but only respond Acade- mies to the requests and survey only in the form of the answer. The author isn't enough time for public insti- tutions and public servants, etc. the organizational reasons. It is one more example of insufficient care of public authorities concerning training of employees and their inability to understand importance of it as addi- tional function. Even in this case it is impossible to mention the subsequent measures. Though the regulatory base for training is identified as a part of the cycle HR, all this still is a serious obstacle for providing that training hasn't been properly developed.
At the same time from the general education system of Academy it is clear that the main resources are spent for process of training. As the world practice shows, only 15 % of results of the training program are used only in a workplace. When forming a national education system it is necessary to pay more attention to increase in this indicator.
Level 1 and 2 of Kirkpatrick can be partially used, the level 3 and level 4 it is possible to avoid, having studied experience of Academy as key supplier in the market of training in civil service (other suppliers of civil services have to consider experience of Academy as an example) It is possible to tell once again that all estimated events are held only by Academy. And, actually, for the government the most important is use of knowledge and skills received during activity and development of public administration. Then there is a
Evaluating effectiveness of the trainingof civil servants
question who and how to estimate the last two levels. Definition of degree of satisfaction with results of training of public authorities can be carried out spontaneously.
As a result of a survey conducted among 47 workers of HR departments of government agencies in September-October 2017, 41,2 % of respondents indicated that they did not change their learning outcomes and continued to workas before. And 73,3 %of them indicate that the effectiveness of training is likely to be used in the service. In addition, the respondents were asked: «What form of reporting, the results of training will be used after your training in the state body?» (Fig. 4) and «Is the level of knowledge and the level of employment that you acquired during training estimated after a certain period of time (for example, 3-6 months later?» (Fig. 5).
Figure 4. Application of reporting forms, learning outcomes after training (%)
Figure 5. Assessment of knowledge gained during training and assessment after work (%) (Designed by the author according to the survey results)
As we saw, the lack of systematic methodology of results of training results in ignorance and even ig- nores results of training.
As a rule, the impressive and positive correlation between degree of satisfaction of participants of the training program and knowledge, skills and abilities of participants is observed. That is, if the participant is
21,6 15,9 6,5
4,8
46,9 4,3
0 5 10 15 20 25 30 35 40 45 50 Public discussion of results of training with
colleagues
Review your enjoyment of your training or questioning your manager/HR departments
The help in management and service of personnel on use of results of training at work
Control of my behavior by the manager after the end of training
No forms Assessment of the manager by results of
training
58,4 18,1
3,5
20
0 10 20 30 40 50 60 70
There is no assessment to anybody Survey of the teaching organization Talk with HR departments Interview with the manager
satisfied with the educational progra positive correlation between the thi gained during work as a result of o the level of use of the skills and skil plemented by educational organizat human resources. Therefore partners practice, but also on the same party grams it is necessary to intensify a developed plan of realization of the to training educational organizations ments, and consulting and supervi providing that all these elements we include the following levels of asses
Figure 6. L
The cycle of vocational trainin needs for training and definition of has the critical criteria and methods gram consists in reduction of a gap and change of the market of training ant.
Curricula and materials can b results are clearly defined. This organization. The search for support the development of the curriculum.
sanctions came into force special st the content and the contingent. In department. In some countries, the many years. This led to an assessme
Coordination of training activ number of trainees and personnel pl with the law. Each institution deter initiative and, if necessary, determin legal rule that these bodies and offi desirable to provide individual ne
am, he will be able to make it in current state. In ird and fourth levels - as the participant of the p overall effectiveness of public authority. There is lls used by public servants during their work. Th tions, and the last two levels are a part of publi ship between interested parties has to be not only y, as in modern practice. For increase in efficienc additional and preformal preparation before form
third level has to be developed for achievement s have to be in close contact with heads of public ising procedures have to continue after compl ere comprehensive, the author considers that use ssment (Fig. 6).
Level of assessment of the effectiveness of training of civil servants (designed by the author)
ng is closed which should be estimated at all lev influence of results of training on the general pu s of assessment.The key question of preliminary
between productivity and productivity by definit g for the solution of practical questions in daily be developed correctly only when the learning can be determined by a special advisory bo t from the consumer-public body is a characterist To determine the need for public service in fore tructural units and procedures. The methods used
many countries, the definition of needs depen demand for training is formalized not only for ent of the need for training and individual develop vities in Kazakhstan is limited only by provid lans of the presidential decree. Training mechani rmines its evaluation mechanisms, for example, nes the needs of state institutions and civil servan
icials are obliged to participate in such a survey eeds, which are assessed by training centers w
n the same way there is a program uses knowledge s no correlation between e first two levels are im- ic institutions, including y unilateral in the current cy and efficiency of pro- mal education. Carefully
of desirable result. Prior c authorities, HR depart- letion of a research.For
of public servants has to
vels, from the analysis of ublic service. Each level y preparation of the pro-
tion of needs for training work of the public serv- objectives and expected ody of the educational tic of his participation in ign countries, legislative d can vary depending on nds on each ministry or r one year, but even for pment plans for staff [9].
ding information on the sms do not fully comply on the Academy's own nts. However, there is no y. Therefore, it would be with promotion of civil
Evaluating effectiveness of the trainingof civil servants
servants, with individual development plans, unless the persons responsible for training can raise it to the appropriate competent authority. An individual plan for professional development of a civil servant is the only means of determining his educational needs. The rationale for introducing it as a rational human resources technology will be discussed in the next section.
Thus, each of assessment procedures of effective requirements has to be based on information provided by public authority and the public servant and it includes constantly happening actions, such as:
– identification of stakeholdersof education;
– diagnostics of demand for training;
– to analyze the obtained data, thereby defining the expected results;
– planning and definition of priorities of training events (together with the organization for training of public authorities).
According to us, the mechanism of selection of thematic offers on creation of the advanced content of training has to consist of two blocks:
1. Requirements and requests of state bodies. The procedure described above is considered sufficient to determine the needs of the authorities.
2. Requirements and requirements of civil servants:
a) Studying the results of strategic planning, assessing the performance of staff, researching work functions, interviews, interviews with directors, direct observation, focus groups, performance evaluation results, analysis of reports, competency profile and performance evaluation;
b) an information base or a system for monitoring the quality of education, as well as information and analytical monitoring of routes should be created and that it will ensure centralized, systematic unity and control all internal processes.
The international practice has shown that such analyses and opportunities have been raised to the level of the government [10], in State department of the United States of America [11] and on government web- sites in United Kingdom, public servants have an opportunity to be registered and hold own consultations on 25 professional networks [12].
After assessment of needs for training the design, formatting, realization and assessment of programs take root. Participants have concrete methods of assessment of education levels which they have got, and satisfactions of the offered educational products.
Their volume, depth, level of knowledge, compliance to qualifications and qualifications of skills and qualification, the number of the studied programs, the nature of knowledge, level of knowledge and skills can be key indicators of level of knowledge acquired by participants. The Scrum technology can be used at this level of training, that is at the beginning of the program participants form backlog and measure the level of access to the program at the end.
In case of formation assessment of satisfaction of public servants with results of training is carried out only by the teaching organization. Such assessment has to be carried out first of all by public authority as key interested party and its HR department. As a result of assessment of these indicators it is possible to define whether the offered training program answers the specified purpose. Methods of assessment can vary de- pending on the size, terms and the contingent of financial resources which can be written or functional test- ing with use of instruments of modeling, etc. At this level it is important to watch participants of the program in absentia, appraisers of this level - teachers, coordinators, and responsible people.
The subsequent measures are directed to control of that, how really programs of training are really car- ried out in a workplace. Assessment of this level defines degree of the effective organization of all previous actions. It is possible to see that in today's Kazakhstan case he doesn't consider importance of combination of efforts for assessment of this level and deepening of the gained knowledge. They are applied only indirectly, using additional elements as it is stated above, post-monitoring and various questionnaires. As well as in a case with the general practice, the value of poll is limited. If inspections are complete at the end of a re- search, results of post-monitoring inspections don't allow to draw reliable conclusions as the coefficient of feedback is too low and based on direct influence of the fact that participants can't provide information on depth of the knowledge and skills. Besides carrying out the questionnaire according to the program, partici- pants can also use polls, an interview, the subsequent sessions, observation of jobs, focus groups, efficiency assessment, results of training, the dashboard. The obtained data can be used as a basis for release of the software.
Formulation of one method and its application to all programs is very misleading interpretation that leads to big mistakes in assessment. Each program has to be adapted for definition of influence of public
servants for their work. Of course, it is resource-intensive and difficult process. Therefore in a present situa- tion there is no corresponding methodology.
All these examples can be used as the sizes of separate programs which can't be considered as the gen- eral indicators for all programs. The most difficult assessment of this level is definition of the corresponding indicators and indicators. Thus, it is possible to reach appropriate level of assessment only by fixing of con- crete indicators according to each program even if it is expensive, from the point of view of expenses on re- sources on the basis of which it is possible to correct contents of the program. Indicators, such as percent of managers which as it is considered, correspond to the expected levels of competence and ability to corre- spond to administrative indicators can be defined as the general indicators of quality.
In this regard the author recommends to carry out the Program for a design cycle as the pilot project and to formulate methodology of assessment of its efficiency. Measures which need to be accepted have to be closely connected with the systems of monitoring and assessment of projects of public authorities, design offices. Thus, the efficiency of investments into the program of project management can be defined in time and money equivalents. Here is how the program productivity indicator can be displayed (Table):
T a b l e Quantitative and quality indicators of efficiency of the programs «Project management»
1 Performance Indicators 2 Intangible benefits 1.1 The number of the projects submitted prior to train-
ing and quantity of financed.
1.2 Number of the involved public servants.
1.3 The allocated funds.
1.4 Size of possible profit.
1.5 Working hours.
1.6 The repeating functions.
Final assessment. Interrelation between volumes of post-processing of the revealed indicators
2.1 Increase in institutional potential 2.2 Improvement of investment planning.
2.3 Increase in skills and opportunities of public servants, degree of satisfaction with work, abili- ties to work in groups.
2.4 Improvement of the information environ- ment for the recommendations of the project.
Note: Designed by the author.
The same way, Strategic planning (where the key indicator submits the reporting under the strategic plan), the program of trainers on leadership of trends also estimates both material, and intangible assets in the pilot project, and then separates from other programs, formulating methodology of other programs, to opti- mize the spent resources and to increase their productivity. In this regard the author suggests to create the contingent of public servants, but not to select participants for separate objects and to develop groups on the basis of groups, such as (Project Office) project management, on the basis of structural divisions. Then pro- ject management, risk management, strategic planning, etc. Of course, the efficiency of programs will in- crease.
Besides, the curriculum and materials have to be confirmed by means of an experimental seminar. Then the expected results carefully are estimated. Especially for «A» and «B» administrative civil servants are ap- proved competent system, have been included in programs of development of competence of the General competences directed on change of the behavior of consumers accepted the level of their realization of public authorities demand need of achievement of very high extent of support. It is necessary to create the organiza- tional and regulatory base for such participation. It doesn't mean that rating indicators have to describe the purposes of the training program completely. Indicators have to have additional «criteria of demand» which will have the value which estimates efficiency of each interested party: the pupil, his / her heads and the sub- ordinated employees and also human resources. If training is focused on providing knowledge instead of be- havioral changes, such as changes in regulatory frameworks or administrative procedures, newer, rule-based indicators will be much easier. You can tell the number of filed or canceled decisions as an available indica- tor. Therefore, the collection of information from trainees and their managers in the staff resources of a state- government body can be the basis of reliable information.
Above-mentioned consists that process of determination of learning efficiency has defined itself as the strategic tool which will help to resolve issues which have to be considered in work of public authorities and