In this study, we introduce a framework designed to help educators assess the effectiveness of popular generative artificial intelligence (AI) tools in solving authentic assessments. We employed Bloom’s taxonomy as a guiding principle to create authentic assessments that evaluate the capabilities of generative AI tools. We applied this framework to assess the abilities of ChatGPT-4, ChatGPT-3.5, Google Bard and Microsoft Bing in solving authentic assessments in economics. We found that generative AI tools perform very well at the lower levels of Bloom's taxonomy while still maintaining a decent level of performance at the higher levels, with “create” being the weakest level of performance. Interestingly, these tools are better able to address numeric-based questions than text-based ones. Moreover, all the generative AI tools exhibit weaknesses in building arguments based on theoretical frameworks, maintaining the coherence of different arguments and providing appropriate references. Our study provides educators with a framework to assess the capabilities of generative AI tools, enabling them to make more informed decisions regarding assessments and learning activities. Our findings demand a strategic reimagining of educational goals and assessments, emphasising higher cognitive skills and calling for a concerted effort to enhance the capabilities of educators in preparing students for a rapidly transforming professional environment.
Implications for practice or policy

Our proposed framework enables educators to systematically evaluate the capabilities of widely used generative AI tools in assessments and assist them in the assessment design process.
Tertiary institutions should re-evaluate and redesign programmes and course learning outcomes. The new focus on learning outcomes should address the higher levels of educational goals of Bloom’s taxonomy, specifically the “create” level.


Race with the machines: Assessing the capability of generative AI in solving authentic assessments

The main purpose of this study is to design a learning outcomes system and assess its importance in academic higher education programs in Colombia. The approach applied is propositional and rational and compares the Decrees 1330 of 2019 and 21795 of 2020 issued by the Ministry of National Education of Colombia. Analyzes are performed from a conceptual perspective to examine the standards of both the National Agency for Quality and Accreditation (ANECA, in Spanish) and the Council for Higher Education Accreditation (CHEA, in Spanish). As a result, criteria and components are evaluated to develop a learning outcomes system that generates an integral perspective. In addition, criteria are established for the operability of learning outcomes (LO) and evaluation mechanisms (EM). In conclusion, a practical operability structure is proposed that allows integrating different academic activities with learning outcomes obtained during the educational process.

http://www.scielo.cl/pdf/formuniv/v15n1/0718-5006-formuniv-15-01-105.pdf

Resultados de aprendizaje y mecanismos de evaluación en los programas académicos de educación superior en Colombia

The advances in large language models (LLMs) are evolving rapidly. Artificial intelligence (AI) chatbots based on LLMs excel in language understanding and generation, with potential utility to transform healthcare education and practice. However, it is important to assess the performance of such AI models in various topics to highlight its strengths and possible limitations. Therefore, this study aimed to evaluate the performance of ChatGPT (GPT-3.5 and GPT-4), Bing, and Bard compared to human students at a postgraduate masters (MSc) level in Medical Laboratory Sciences. The study design was based on the METRICS checklist for the design and reporting of AI-based studies in healthcare. The study utilized a dataset of 60 Clinical Chemistry multiple-choice questions (MCQs) initially conceived for assessment of 20 MSc students. The revised Blooms taxonomy was used as the framework for classifying the MCQs into four cognitive categories: Remember, Understand, Analyze, and Apply. A modified version of the CLEAR tool was used for assessment of the quality of AI-generated content, with Cohens {kappa} for inter-rater agreement. Compared to the mean students score which was 40/60 (66.8%), GPT-4 scored 54/60 (90.0%), followed by Bing (46/60, 76.7%), GPT-3.5 (44/60, 73.3%), and Bard (40/60, 66.7%). Statistically significant better performance was noted in lower cognitive domains (Remember and Understand) in GPT-3.5, GPT-4, and Bard. The CLEAR scores indicated that ChatGPT-4 performance was "Excellent" compared to "Above average" performance of ChatGPT-3.5, Bing, and Bard. The findings indicated that ChatGPT-4 excelled in the Clinical Chemistry exam, while ChatGPT-3.5, Bing, and Bard were above-average. Given that the MCQs were directed to postgraduate students with a high degree of specialization, the performance of these AI chatbots was remarkable. Due to the risks of academic dishonesty and possible dependence on these AI models, the appropriateness of MCQs as an assessment tool in higher education should be re-evaluated.

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5, and Humans in Clinical Chemistry Multiple-Choice Questions

This study validates the recommendations proposed by Tumpa et al. (2022a) for designing authentic group-based assessments in order to improve the work-ready attributes of project management graduates. The study aims to address the concerns of employers who have reported dissatisfaction with the employability skills of project management graduates. Two focus groups, comprising of fifteen project management academics from Australian and UK universities, were conducted to gather data for this study. The findings of the focus groups not only validated the majority of the recommendations but also provided new insights into the design and administration of group-based assessments for project management education. The study sought to contribute to the improvement of skills of project management graduates by providing a valid and authentic approach to group assessment design for project management academics to use. The implications of this study are that it provides an evidence-based framework for designing group-based assessments, which in turn will benefit both the graduates and the employers. 

Enhancing project management graduates’ employability through group assessment innovations: An empirical study

Artificial intelligence (AI) chatbots excel in language understanding and generation. These models can transform healthcare education and practice. However, it is important to assess the performance of such AI models in various topics to highlight its strengths and possible limitations. This study aimed to evaluate the performance of ChatGPT (GPT-3.5 and GPT-4), Bing, and Bard compared to human students at a postgraduate master's level in Medical Laboratory Sciences. 

Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5 and Humans in Clinical Chemistry Multiple-Choice Questions

\nPurpose\nThis study aims to investigate how to improve the project management (PM) curriculum by evaluating the nature and alignment of learning outcomes in the PM course syllabi with Bloom’s Taxonomy framework.\n\n\nDesign/methodology/approach\nThe research methodology for this study is an integrative approach that uses document analysis and content analysis. The data set was selected based on a purposeful sampling method and came from PM course syllabi for classes that were taught during the 2016–2018 academic years.\n\n\nFindings\nResults revealed that most of the reviewed PM course syllabi contained learning outcomes although they were written and assessed at the lower levels of Bloom’s Taxonomy and knowledge dimensions. The study calls for the academy and industry to partner in improving the PM curriculum to lower the PM talent deficit and increase project success rates.\n\n\nResearch limitations/implications\nThe absence of PM learning outcomes or the presence of poorly written PM learning outcomes in a course implies that the academy should provide professional development programs to help professors learn how to formulate and write specific, measurable, achievable, relevant and timely learning outcomes. The professors should also ensure that the learning outcomes use a type of cognitive taxonomy that is aligned with the appropriate assessments to measure, monitor and guarantee assurance of learning.\n\n\nPractical implications\nAcademy and industry partners can work collaboratively to provide students with opportunities that expose them to real-world experiential projects, internships and job opportunities while concurrently giving them hands-on practical applications of learned PM knowledge and skills. The society will be well served when the academy is able to produce well-qualified PM personnel capable of successfully carrying out PM activities and lowering the project’s failure rates.\n\n\nSocial implications\nThe society will be well served when the academy is able to produce well-qualified PM personnel capable of successfully carrying out PM activities and lowering the project’s failure rates.\n\n\nOriginality/value\nTo the researchers’ knowledge, this is the first study to specifically investigate the presence and nature of PM learning outcomes in course syllabi. By evaluating the alignment between PM learning outcomes and Bloom’s Taxonomy action verbs and cognitive processes, the study provides some exemplars of well-written and measurable learning outcomes that professors can use to inform their PM curriculum through course design or redesign.\n

Improving Project Management Curriculum by Aligning Course Learning Outcomes with Bloom's Taxonomy Framework.

PurposeAlthough project management (PM) continues to rise in popularity, there is still a significant PM talent deficit, leading to more challenged or failing projects. To lower the PM talent deficit and mitigate the higher project failure rates, academic institutions have been developing PM curriculums aimed at inculcating a repertoire of competencies to the potential project managers. In developing an ideal well-rounded PM curriculum, academic institutions occasionally engage the input of industry partners and governing entities. The study aims to (1) compare the competencies in one of the leading industry competency model and framework (PMI Talent Triangle) to the competencies in the PM course syllabi learning outcomes, (2) determine the extent to which these two sets of PM competencies are aligned and (3) and explore avenues for improvements.Design/methodology/approachThe study uses a purposeful sampling method to gather PM course syllabi. The PM competencies data are gleaned from the syllabi using the content analysis method. Thereafter, QSR NVivo qualitative statistical software is used to summarize and analyze the competency data from the learning outcomes.FindingsThe results reveal that most of the PM competencies in the course syllabi fall under the technical PM domain. Specifically, the top three competency elements in each domain are technical PM domain (PM skills, tools and techniques, schedule management and cost estimation/budget), leadership domain (team-building, verbal/written communication and problem-solving) and strategic and business management domain (strategic planning, analysis and alignment, benefits management and realization, customer relationship and satisfaction).Research limitations/implicationsThe study investigates the alignment of the PM course competencies with competency domains in the PMI Talent Triangle, a global competence model that is well aligned with other global competence models such as the APM Competence Framework, the ICB4 Individual Competence Baseline and the PROMA3.Practical implicationsThe results from this study provide guidelines useful in informing PM curricula re/design, as well as the inculcation of knowledge, skills, tools, techniques and behaviors needed for effective PM.Social implicationsThe PM curriculum can be improved by partnering with PM industry leaders who can serve as advisors to the academy on industry needs, direction and emerging innovations that can inform PM learning outcomes, PM curricular design and the development of quality PM talent. The academy and the industry are encouraged to actively strive for mutual partnerships where PM professionals and academicians serve on each other's advisory boards. Also, the academy can partner with the industry professionals by developing curriculum resources such as case studies that bring the real-life PM applications to the classroom.Originality/valueThis study is motivated by the call for research studies that provide a holistic picture of the desired PM competencies and an exploration and definition of the educational needs in the PM curriculum.

The role of industry and academia partnership in improving project management curriculum and competencies

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Laurell C. Malone

Author Tools

Chat about Author

Papers

Improving Project Management Curriculum by Aligning Course Learning Outcomes with Bloom's Taxonomy Framework.

The role of industry and academia partnership in improving project management curriculum and competencies