OUTCOMES ASSESSMENT IN MINING ENGINEERING Mary Poulton Head, MGE Department
ABET Accreditation Board for Engineering and Technology 9 criteria evaluated: Students Program educational objectives Program outcomes and assessment Continuous improvement Curriculum Faculty Facilities Support Program criteria (established by professional societies)
CONTINUOUS IMPROVEMENT Must demonstrate that we close the loop on continuous improvement during 6 year interim period Cannot accredit and forget it Document everything Process must become part of department culture
CURRICULUM INTEGRATION AND ASSESSMENT PROJECT - 2006 identify the goals of our curriculum (program outcomes for ABET) identify what we teach and why (spreadsheet) identify how our course content fits together (how do we cluster courses/modules/topics and how do we sequence them) identify our desired outcomes for courses (what should a student know and be able to do by the end of a course) develop an assessment for the outcomes assess our courses and curriculum evaluate whether our classes and curriculum meet our outcomes assess whether students are learning evaluate how our course delivery and assignments and linkages can improve learning repeat steps 1-9 to improve each course, track, overall curriculum
PROGRAM CRITERIA (SME) Graduates of the program must have demonstrated: a) A b i l i t y t o b e a b l e t o a p p l y m a t h e m a t i c s t h r o u g h d i f f e r e n t i a l e q u a t i o n s, c a l c u l u s - b a s e d p h y s i c s, g e n e r a l c h e m i s t r y, a n d p r o b a b i l i t y a n d s t a t i s t i c s a s a p p l i e d t o m i n i n g e n g i n e e r i n g p r o b l e m s a p p l i c a t i o n s ; b) F u n d a m e n t a l k n o w l e d g e i n t h e g e o l o g i c a l s c i e n c e s i n c l u d i n g c h a r a c t e r i z a t i o n o f m i n e r a l d e p o s i t s, p h y s i c a l g e o l o g y, s t r u c t ural o r e n g i n e e r i n g g e o l o g y, a n d m i n e r a l a n d r o c k i d e n t i f i c a t i o n a n d p r o p e r t i e s ; c ) P r o f i c i e n c y i n s t a t i c s, d y n a m i c s, s t r e n g t h o f m a t e r i a l s, f l u i d m e c h a n i c s, t h e r m o d y n a m i c s, a n d e l e c t r i c a l c i r c u i t s. d ) P r o f i c i e n c y i n e n g i n e e r i n g t o p i c s r e l a t e d t o b o t h s u r f a c e a n d u n d e r g r o und m i n i n g i n c l u d i n g : m i n i n g m e t h o d s, p l a n n i n g a n d d e s i g n, g r o u n d c o n t r o l a n d r o c k m e c h a n i c s, h e a l t h a n d s a f e t y, e n v i r o n m e n t a l i s s u e s, a n d v e n t i l a t i o n. e) P r o f i c i e n c y i n a d d i t i o n a l e n g i n e e r i n g t o p i c s s u c h a s r o c k f r a g m e n t a t i o n, m a t e r i a l s h a n d l i n g, m i n e r a l o r c o a l p r o c e s s i n g, m i n e s u r v e y i n g, a n d v a l u a t i o n a n d r e s o u r c e / r e s e r v e e s t i m a t i o n a s a p p r o p r i a t e t o t h e p r o g r a m o b j e c t i v e s. f) T h e l a b o r a t o r y e x p e r i e n c e m u s t l e a d t o p r o f i c i e n c y i n g e o l o g i c c o n c e p ts, r o c k m e c h a n i c s, m i n e v e n t i l a t i o n, a n d o t h e r t o p i c s a p p r o p ri a t e t o t h e p r o g r a m o b j e c ti v e s.
Skills 296a 396a 419 412 Capstone: 498 Processing: 411, 439 Environmental: 422, 441 Mineralogy: 210 Valuation: 430 Surface: 427 415 434 Underground: 438 406A 447 Integrative: 200, 426, 421, 407, 409
PROGRAM OBJECTIVES WHAT AN ALUM CAN DO 3-5 YEARS AFTER GRADUATION A practicing mining engineer: 1. Can tie the first principles of engineering with computer -based solutions to validate computer output, understand the difference between theoretical and practical solutions. 2. Can conduct economic and risk analyses; understand a business plan and responsibilities to customers, stockholders, and stakeholders. 3. Can stay current with technology and industry practices. 4. Can effectively communicate with peers, front -line workforce, and management; possess the skills to be a team player. 5. Takes safety and environmental concerns into consideration in designs. 6. Can understand the human and social elements of a mining operation and its importance, dynamics, and sensitivity to internal stimuli as it drives the safety, costs, and productivity of the operation. 7. Possess the ability to organize, plan, and schedule projects to effectively manage resources and reach deadlines.
LEARNING OUTCOMES WHAT A STUDENT CAN DO UPON GRADUATION A: Demonstrate proficiency in mathematics through differential equations; physics including mechanics, thermodynamics and circuits; basic chemistry principals and laboratory techniques; basic geosciences; probability and statistics; including the ability to conduct experiments and analyze data. B: Be able to complete a design project including elements that emphasize project management, supervision, and effective communication.
LEARNING OUTCOMES C: Be able to solve engineering analysis problems with increasing dif ficulty through the curriculum, including open - ended problems and the impact of the solution on safety and quality D: Prepare technical reports (including team based) in written form including graphs and tables, and oral reports in prepared presentations; identify the need for information, locate the information, assess the quality of the information, and use the information effectively.
LEARNING OUTCOMES E: Demonstrate sensitivity to issues in humanity F: Be involved in professional societies, outreach, or research. G: Integrate computers and software to solve engineering problems and have a working knowledge of mining equipment/tools.
MAP MNE OUTCOMES TO ABET OUTCOMES THE NOTORIOUS A-K a) an ability to apply knowledge of mathematics, science, and engineering b) an ability to design and conduct experiments, as well as to analyze and interpret data c) an ability to design a system, component, or process to meet desired needs d) an ability to function on multi-disciplinary teams e) an ability to identify, formulate, and solve engineering problems f) an understanding of professional ethical responsibility g) an ability to communicate effectively h) the broad education necessary to understand the impact of engineering solutions in a global and societal context i) a recognition of the need for, and an ability to engage in, life -long learning j) a knowledge of contemporary issues k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice
MAP OUTCOMES TO OBJECTIVES AND TO PROGRAM ACTIVITIES Create matrix to do mappings Objectives measurements Industry Leadership Board Meetings with industry, government, NGOs, and alumni (state mine inspector, MSHA, FMI health, BLM, Labor Union) Feedback from employers for summer hires Near Term Alumni Survey (1-5 years out) Faculty meetings Academic Program Review (conducted every 7 years and includes external and internal reviewers)
DEFINE ACTIVITIES TO MEASURE OUTCOMES 1. Near-Term Alumni Survey 2. Graduating Senior Surveys 3. Faculty review 4. ILB Audit Committee evaluation 5. Course outcomes worksheets 6. Student performance in foundational courses
COURSE OUTCOMES WORKSHEETS Every course, every semester Fill out Excel spreadsheet that maps every assignment, lab, exam, etc to our MNE Outcomes Roll all courses into an analysis of attention paid to each outcome, overall student performance per outcome, use for feedback Parse transcripts so we have all grades in non-mne courses
ALUMNI RESULTS FOR OBJECTIVES Our alumni reported that: 94% felt they were prepared for their first job 94% were mostly or highly satisfied that their education met their career needs (avg rating = 4.6/5.0) 94% said that a professional engineering license was not important for their current job; 25% had taken and passed the FE exam 100% were members of SME as students 88% participated at the San Xavier Mine as undergraduates 50% participated in a research project with a faculty member as an undergraduate 25% have worked internationally since graduation
Table 3-3: Course outcome coverage 2009 Course Outcome A Outcome B Outcome C Outcome D Outcome E Outcome F Outcome G MNE 200 42% H 51% H 8% L MNE 210 94% H 6% L MNE 296 MNE 396 100% H 100% H MNE 406 12% L 8% L 57% H 17% M 5% L 2% L MNE 407 30% H 46% H 18% M 6% L MNE 411 20% L 80% H MNE 412 17% L 72% H 12% L MNE 415 7% L 10% L 61% H 13% L 10% L MNE 419 100% H MNE 422 16% L 30% H 46% H 8% L MNE 426 1% L 89% H 10% L MNE 427 17% M 55% H 16% L 12% L MNE 430 27% H 10% L 35% H 17% M 11% M MNE 434 8% L 75% H 9% L 8% L MNE 438 29% H 33% H 26% H 12% M MNE 447 2% L 33% H 33% H 8% L 1% L 23% H MNE 498 50% H 40% H 10% L
COURSE OUTCOMES ANALYSIS initial goal of 80% of our students achieving a score of 75% (C) or better for each outcome for each class.
Table 3-4. Percentage of students scoring 75% or higher on each outcome in 2009. Goal is 80% of students score 75% or higher. Course OUTCOME A OUTCOME B OUTCOME C OUTCOME D OUTCOME E MNE 200 94% 73% 94% 85% MNE 210 95% 95% MNE 296 82% MNE 396 71% OUTCOME F OUTCOME G MNE 406 87% 100% 87% 87% 100% 87% MNE 407 80% 47% 67% 47% MNE 411 100% 100% MNE 412 64% 86% 64% MNE 415 79% 96% 79% 96% 96% MNE 419 100% MNE 422 72% 52% 55% 83% MNE 426 76% 94% 59% MNE 427 71% 71% 83% 71% MNE 430 100% 100% 100% 100% 100% 100% MNE 434 43% 73% 73% 43% MNE 438 57% 43% 86% 86% MNE 439 100% 100% MNE 447 80% 100% 87% 97% 100% 97% MNE 498 86% 76% 90% AVERAGE 85% 83% 80% 81% 83% 90% 79%
Table 3-8. Graduating Senior Survey analysis on a 5 pt scale. n=2 n=3 n=19 n=9 n=11 Outcome 2006 2007 2008 2009 2010 Average A Foundation courses 3.6 3.4 3.4 3.5 3.2 3.4 B Design C Engr analysis D Communica tion E Humanity F Professional ism G Modern Tools 3.3 4.2 3.9 3.8 4.1 3.9 3.6 4.1 4.2 4.0 4.1 4.1 3.8 3.8 3.9 3.8 4.2 3.9 3.5 4.2 4.0 4.1 4.2 4.0 4.2 4.4 4.3 4.1 4.2 4.2 3.5 4.3 4.0 4.1 4.0 4.0
Table 3-9: Mining GPA Performance by Calendar Year in Foundation Courses. Mean GPA 2004 2005 2006 2007 2008 2009 Cumulative 2.998 3.016 2.799 3.421 3.034 2.930 Lower Division 3.060 2.956 2.983 3.192 3.022 2.954 Upper Division 3.207 3.233 3.061 3.719 3.365 3.179 Mathematics 2.732 2.394 2.747 2.790 2.701 2.660 Physics & Chemistry 2.807 2.459 2.715 2.907 2.911 2.773 General Education 3.337 3.364 3.265 3.211 3.359 3.187
FACULTY EVALUATION Example rubrics Rating 1 2 3 4 5 Rubric Outcomes are defined but no tools used to evaluated them Non-quantitative tools are used but results are not used to improve the program Quantitative and qualitative tools are used to evaluate and assess performance on outcomes Benchmarks are established for assessment Formal process for assessment as in 4 and feedback loop is implemented to improve program based on assessment
CLOSING THE LOOP After 2004 review we made major curricular changes Between 2006 and 2010 we evaluated the changes and compared to 2004 data Based on evaluation, we made interim tweaks After 2010 review we are again making significant changes teaming, design, tools We benchmark against other universities as well as ourselves Mitigating factors are budget, faculty, facilities Faculty talking to each other about courses is important part of the evaluation
IS IT WORTH IT? Focuses us on quality of undergraduate education Provides motivation to improve courses But Get assessment burn out especially for 1 year after review Expensive Time consuming Inconsistent interpretations and changes with time Need to find way to build into faculty annual evaluation
QUESTIONS?