Assessment Report for GPC By The College Assessment Committee, HCoB March 2010
The Assessment Committee has received data, analyses, and executive summaries of assurance of learning (AOL) assessments for undergraduate core courses completed in the 2009 calendar year. An executive summary for each course contains action(s) recommended by the department responsible for teaching that course. A summary of the actions arranged in the order of direct learning objectives (DLOs) is presented on the following page. The Assessment Committee strongly encourages the GPC to discuss the report within your committee and to have members carry that conversation back to departments. Questions to be asked include, What have we learned from assessment that needs to be disseminated to all faculty teaching within the MBA? Where do our students show strong evidence of learning? Where do our students show limited evidence of learning? Now that the college has demonstrated to the AACSB that we have closed the loop this would be a good time to revisit learning goals and objectives of the MBA. Do the current DLOs reflect the foundation of our MBA? Do the metrics make sense? Do we have too many learning objectives? Too few? In which courses should assessments take place? It is also important to consider the frequency with which assessments are made. Per the current framework, assessments should be conducted each semester the course is offered until an approved schedule of assessments is completed. The Assessment Committee also recommends the GPC review benchmarks that have been established for each DLO. Benchmarks currently range from a low of 60% to a high of 95%. Would it make sense to standardize the benchmarks or establish a range of benchmarks for the MBA? Answering these questions will require the UPC to consider the rationale for having such different benchmarks for DLOs within the program. 2 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO1: [ACTY 6110, FIN 6120, MKTG 6130, MGMT/MKTG 6140, BUS 6180] MGMT/MKTG6140: The mechanism for this measure (Calculate & interpret quality control charts including X-bar and R- charts) may need to be refined. In some cases students correctly calculated and interpreted the data. In other cases, students may have made a minor error in calculation but correctly interpreted the results they (incorrectly) developed. A discussion should be held to determine if the calculation or the interpretation should be the primary measure for the MBA program. met in all other courses. No actions needed. DLO2: [BUS6150] met. No action needed. DLO3: [MGMT/MKTG 6140, BUS 6990, BUS 6180] met in MGMT/MKTG 6140. No actions needed. Rewrite lecture material in BUS 6990 on suboptimization across functions and find an article on the topic to distribute to class. Additional review, including application is needed for balance scorecards. BUS 6180 did not conduct assessments for DLO 3 in 2009. Further, the course coordinator feels BUS 6180 is not an appropriate class to be assessing this DLO and the assessment plan should be modified accordingly. DLO4: [BUS 6150, BUS 6160, MGMT 6170] Future efforts of assessment in MGMT 6170 may want to assess international/english as a second language students separately from students whose primary language is English, since the assessment is about written communication skills. met in all other courses. No action needed. DLO5: [BUS 6150, BUS 6160] met. No action needed. DLO6: [FIN 6120, BUS 6180] met in FIN 6120. No action needed. Class discussion in BUS6180 will focus more on solving IT problems presented in the case studies. 3 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
Executive Summary [Graduate s] 4 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
Assessment Metrics 5 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 1: Demonstrate a solid conceptual foundation in each of the functional areas of business. Benchmark ACTY 6110 Ruhl SP 2009 75 85.9 24 Yes 1. Two areas were assessed: service costing and ACTY 6110 Ruhl SP 2009 75 79.5 24 Yes activity-based costing. The assessment was done via exam questions. scored an average of 85.9% on the service costing exam question. No further action was taken, as the benchmark was met. 2. The other area assessed in Spring 2009 was activity-based costing. scored an average of 79.5% on the service costing exam question. No further action was taken, as the benchmark was met. ACTY 6110 Ruhl F2009 75 90.0 36 Yes This was a job order costing question assessed using an exam question. The benchmark was 75%, the mean was 90%. No action taken.job Order Costing - Exam Question (Exam 1 Question 4) N=36, Range 5-15, Mean=90%. ACTY 6110 Ruhl F2009 75 90.0 36 Yes This was a cost allocation problem that was part of a case. The benchmark was 75% and the actual mean percent achieved was 90.0%. No action taken. 6 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 1: Demonstrate a solid conceptual foundation in each of the functional areas of business. FIN6120 Yaman Benchmark SP 2009 60 80 34 Yes The department has done a full cycle of SUII 2009 60 93 33 Yes 2009 summary 75 87 67 Yes assessment of DLO#1 and the results have been discussed with faculty. Based on faculty input, the department has decided not to reallocate time spent on individual components of DLO#1. 7 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 1: Demonstrate a solid conceptual foundation in each of the functional areas of business. MKTG6130 Schultz F 2009 80 87 22 Yes Update met skills. MKTG6130: The quiz varies in difficulty each semester. need additional work in this area. One option has been to have the student spend an extra week going over the material and resubmitting the quiz. The revised scores indicate the students are learning the material with additional time. MKTG6130 Schultz F 2009 80 90 22 Yes Update met skills MKTG6130: exceeded expectations in all semesters that data were collected. Overall, results suggest that the students are learning the principles listed. Update - Faculty plan to designate sustainability topics covered in class to support the college Center for Sustainability. Marketing analytics Benchmark MKTG6130 Schultz SP 2009 80 90 24 Yes Market analysis: Benchmark MKTG6130 Schultz SP 2009 80 94.5 24 Yes Segmentation Benchmark MKTG6130 Schultz SP 2009 80 88.5 24 Yes MKTG6130 Schultz F 2009 80 87 22 Yes Update skills met. 8 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
MKTG6130: exceeded expectations in all semesters that data were collected. Overall, results suggest that the students are learning the concepts of Segmentation, Targeting and Positioning. MKTG6130 Schultz F 2009 80 92 22 Yes Update students met skills MKTG6130: exceeded expectations in all semesters that data were collected. Overall, results suggest that the students are learning the strategies. Marketing Mix Strategies Benchmark MKTG6130 Schultz SP 2009 80 94.5 24 Yes Value, Satisfaction, Loyalty Benchmark MKTG6130 Schultz SP 2009 80 88 24 Yes MKTG6130 Schultz F 2009 80 88 22 Yes Update students met skills. MKTG6130: exceeded expectations in all semesters that data were collected. Overall, results suggest that the students are learning the concepts of value, satisfaction and loyalty. 9 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 1: Demonstrate a solid conceptual foundation in each of the functional areas of business. Measure 1: Quality control: Calculate & interpret quality control charts (e.g., X-bar and R-charts); Project, case or exam question. Measure 2: Quality control: Calculate & interpret process capability (e.g., Cp, Cpk, Sigma level); Project, case or exam question Student s MGMT 6140 Scannell FA2009 75 82% 29 Yes Continue with current methods. MGMT 6140 Golhar SP2009 75 78% 23 Yes Measure 3: Capacity Analysis: Determine capacity of a manufacturing or service production process; Project, case or exam question MGMT 6140 MGMT 6140 Benchmark MGMT 6140 Scannell F2009 75 79% 29 Yes The mechanism for this measure may need to be refined. In some cases students correctly calculated and interpreted the data. In other cases, students may have made a minor error in calculation but correctly interpreted the results they (incorrectly) developed. A discussion should be held to determine if the calculation or the interpretation should be the primary measure for the MBA program. MGMT 6140 Golhar SP2009 75 83% 23 Yes Benchmark Benchmark Student s Scannell FA2009 75 87% 29 Yes The initial assessment (exam one) yielded an approximate pas rate of 68%. Re-assessment (final exam) yielded higher results (87%). Golhar SP2009 75 95% 23 Yes 10 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 1: Demonstrate a solid conceptual foundation in each of the functional areas of business. Benchmark BUS6180 BUS6180 Katerattanakul Katerattanakul Spring 2009 F 2009 70 70 90 89 32 32 Yes Yes Action Items listed below: 1. More emphasis was placed on business intelligence and innovation. 2. We provided three exercises on business process and business intelligence. We also issued one exam question about technological innovation. 11 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 2: Demonstrate the capability to function in a global, intercultural, and diverse environment. Bench-mark BUS 6150 Ruhl F2009 If the overall mean on the metrics used is at least 85% (BA) and 75% of the students have an 80% (B) or better score, then MBA students have an acceptable level of performance.. AVG = 87.5 Range = 77.5 92.5 16 Yes None The metric: Understand & appreciate the differences of the global, multi-cultural environment & how to manage within them: Exams questions met. No action needed. 12 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO3: Understand the business as an integrated system and apply strategic planning tools and techniques to coordinate among the functional areas. Measure: Process Analysis: Develop a current state and future state diagram of an integrated process using one or more of the following techniques: process map, flow chart, value stream map; Project, case or exam question Benchmark MGMT 6140 Scannell F 2009 75 100% 29 Yes A half-semester project in which students map an actual business process is used. Given that each situation is unique and that the professor cannot be familiar with the inner workings of every process across every industry, these assignments are graded relative to the proper use of the process mapping rather than whether or not the future state is indeed a feasible option for improvement. MGMT 6140 Golhar SP2009 75 100% 23 Yes 13 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 3: BUS 6990 BUS 6990 Understand the business as an integrated system and apply strategic planning tools and techniques to coordinate among the functional areas. by Benchmark Palmer Spring Flanagan 80 90 28 Yes 2009 Palmer Fall 2009 Flanagan 80 79 28 No Rewrite lecture to drive home the definition of suboptimization across functions and find an article on the topic to distribute to class Knowledge of balanced scorecard fell off between the first exam and the final. Need to review and apply the concept later in the class. 14 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 4: Exhibit business-related behavioral skills including leadership, interpersonal, communication (written and oral), team, and lifelong learning skills. Bench-mark BUS 6150 Ruhl F 2009 If the overall mean on the metrics used is at least 80% (B) and 75% of students have an 80% (B-) or better score, then MBA students have an acceptable level of performance. BUS 6150 Ruhl F 2009 If the overall mean on the metrics used is at least 80% (B) and 75% of students have an 80% (B-) or better score, then MBA students have an acceptable level of performance. AVG = 83 Range = 77.5-90 16 Yes None Exhibit effective (written and oral) communication skills on global issues: Individual Research Paper AVG = 85 16 Yes None: Exhibit effective team skills: Group Presentations & Research Topics met. No action needed. 15 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 4: Exhibit business-related behavioral skills including leadership, interpersonal, communication, team, and lifelong learning skills. Benchmark BUS6160 Hawker Sp 2009 80 93 35 Yes met. No action needed. 16 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 4: Exhibit business-related behavioral skills including leadership, interpersonal, communication, team, and lifelong learning skills. Benchmark MGMT 6170 Stamper Fall 2009 95 93 36 No In prior Fall semesters, students given the same assignment did not have trouble meeting the benchmark criteria of 95%. For Fall 2009, upon further investigation, there appear to be a larger number of international students included in the assessment than in the past. These students are using English as their second language. In this class, there were three international students who did not have strong command of the English language, and did not use the Writing Center to help facilitate better writing on the assignment. These three students were the main contributors to the failure to reach benchmark on this assessment. Future efforts may want to assess international/english as a second language students separately from students whose primary language is English, since the assessment is about written communication skills. 17 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 5: Understand business ethics, social responsibility, and legal implications of business decisions. BUS 6150 Bench-mark Ruhl FA 2009 If the overall mean on the metrics used is at least 85% (BA) and 75% of students have an 80% (B) or better score, then MBA students have an acceptable level of performance. AVG = 88 Range = 82.5-90 16 Yes None. The metric: Understand the link between social responsibility and ethical and legal issues of business in a global environment: Cases & Class Exercises met. No action needed. 18 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 5: Understand business ethics, social responsibility, and legal implications of business decisions. Benchmark BUS6160 Hawker Sp 2009 80 97 35 Yes No action needed 19 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 6: Apply technologies in decision making processes and create innovative business solutions. FIN6120 Yaman Benchmark SP 2009 75 95 34 Yes SUII 2009 75 89 33 Yes 2009 summary 75 92 67 Yes The department has done a full cycle of assessment of DLO#6 and the results have been discussed with faculty. The criteria for this DLO have been met. The overall benchmark for DLO #1 is 75% while each component for DLO#1 has a benchmark of 60%. For DLO#6 the overall benchmark is 75% and there are no benchmarks for individual components. Each DLO has three components measured in each of the semesters and one of the summer sessions. 20 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0
DLO 6: Apply technologies in decision making processes and create innovative business solutions. Benchmark BUS6180 Katerattanakul SP 2009 70 94 32 Yes Action Items listed below: BUS6180 Katerattanakul F 2009 70 94 32 Yes 1. Class discussion in BUS6180 will focus more on solving IT problems presented in the case studies 2. We added seven case studies for students to prepare their case study reports and class discussion. 21 A s s e s s m e n t R e p o r t f o r G P C, M a r c h 2 0 1 0