TL;DR: Co-ML fosters collaborative dataset design practices through a multi-user model building experience, empowering beginners to learn and apply data design practices.
Abstract: Machine learning (ML) models are fundamentally shaped by data, and building inclusive ML systems requires significant considerations around how to design representative datasets. Yet, few novice-oriented ML modeling tools are designed to foster hands-on learning of dataset design practices, including how to design for data diversity and inspect for data quality. To this end, we outline a set of four data design practices (DDPs) for designing inclusive ML models and share how we designed a tablet-based application called Co-ML to foster learning of DDPs through a collaborative ML model building experience. With Co-ML, beginners can build image classifiers through a distributed experience where data is synchronized across multiple devices, enabling multiple users to iteratively refine ML datasets in discussion and coordination with their peers. We deployed Co-ML in a 2-week-long educational AIML Summer Camp, where youth ages 13–18 worked in groups to build custom ML-powered mobile applications. Our analysis reveals how multi-user model building with Co-ML, in the context of student-driven projects created during the summer camp, supported development of DDPs including incorporating data diversity, evaluating model performance, and inspecting for data quality. Additionally, we found that students’ attempts to improve model performance often prioritized learnability over class balance. Through this work, we highlight how the combination of collaboration, model testing interfaces, and student-driven projects can empower learners to actively engage in exploring the role of data in ML systems.
TL;DR: Perancangan ulang website sekolah untuk meningkatkan learnability dan memorability menghasilkan peningkatan yang signifikan.
Abstract: Dalam dunia pendidikan saat ini, website sekolah merupakan suatu alat yang sangat penting dalam memberikan segala informasi kepada masyarakat seperti profil sekolah, prestasi, kegiatan akademik, alumni dan sebagainya. Objek penelitian dalam penelitian ini adalah Sekolah Menengah Atas berbasis Islam Al Irsyad yang berada di Surabaya, dimana telah memiliki website namun belum maksimal dalam penggunaannya sehari-hari karena jumlah pengunjung masih sedikit, tampilan dan fungsi-fungsi fitur dalam website kurang dapat dipahami pengguna. Berdasarkan hasil pengujian usability testing terlihat bahwa dua indikator yakni learnability dan memorability memiliki nilai yang rendah. Oleh sebab itu, tujuan studi ini adalah perancangan ulang desain website sekolah agar mudah diingat dan digunakan pengguna. Metode yang digunakan adalah User Centered Design karena selalu mengikutsertakan penggunanya dalam proses perancangan. Hasil penelitian menunjukkan total rata-rata keseluruhan indikator usability adalah 3.37, hal ini menunjukkan bahwa redesign website sekolah yang diuji berada pada rentang sangat baik, untuk indikator learnability dan memorability sendiri terjadi peningkatan yang signifikan. Kontribusi utama penelitian ini adalah penggunaan usability testing menjadi langkah awal evaluasi kegunaan website untuk meningkatkan antarmuka pengguna.
TL;DR: Wordwall.net is an effective online educational tool for mathematics learning, enabling interactive learning and supporting remote learning.
Abstract: Background: Schools were compelled to switch to distance learning due to community quarantine measures and lockdowns imposed during the global health crisis. In this context, Wordwall.net emerged as a valuable platform, enabling users to independently create interactive games, thereby supporting the remote learning process.Aim: The purpose of this study is to demonstrate how online teaching tools such as Wordwall.net can help teachers enable interactive learning when used correctly.Method: This study employs a cross-sectional analysis to evaluate the effectiveness, efficiency, and user satisfaction of utilizing Wordwall.net as a learning medium for Basic Integration material. This approach encompasses both classroom and online learning, with a particular focus on trigonometry as the final topic mandated by the curriculum. A total of 268 randomly selected volunteer students participated in this research.Result: Using Wordwall.net as a teaching resource for fundamental integration meets the needs of all students, regardless of their background. Furthermore, the findings provide insight into the instructional material's usability in terms of efficiency, efficacy, and satisfaction.Conclusion: Wordwall.net was effective in teaching the essential integration lesson. Therefore Wordwall.net allows teachers to create interactive games and printed classroom resources.
TL;DR: Researchers develop a data-agnostic learnability bound for kernel ridge regression and Gaussian processes, leveraging eigenvalues and eigenfunctions from idealized data-measures to bound sample complexity on realistic data, particularly for natural language processing tasks.
Abstract: Kernel ridge regression (KRR) and Gaussian processes (GPs) are fundamental tools in statistics and machine learning with recent applications to highly over-parameterized deep neural networks. The ability of these tools to learn a target function is directly related to the eigenvalues of their kernel sampled on the input data. Targets having support on higher eigenvalues are more learnable. While kernels are often highly symmetric objects, the data is often not. Thus kernel symmetry seems to have little to no bearing on the above eigenvalues or learnability, making spectral analysis on real-world data challenging. Here, we show that contrary to this common lure, one may use eigenvalues and eigenfunctions associated with highly idealized data-measures to bound learnability on realistic data. As a demonstration, we give a theoretical lower bound on the sample complexity of copying heads for kernels associated with generic transformers acting on natural language.
TL;DR: A computational study demonstrates that natural concepts are more easily learnable, supporting a recent proposal that optimally designed similarity spaces facilitate learning, with evidence presented in the context of perceptual color space.
Abstract: According to a recent proposal, natural concepts are represented in an optimally designed similarity space, adhering to principles a skilled engineer would use for creatures with our perceptual and cognitive capacities. One key principle is that natural concepts should be easily learnable. While evidence exists for parts of this optimal design proposal, there has been no direct evidence linking naturalness to learning until now. This article presents results from a computational study on perceptual color space, demonstrating that naturalness indeed facilitates learning.
TL;DR: This usability testing study of the IPB Help Center Website found satisfactory user experience with high ratings for learnability, efficiency, memorability, errors, and satisfaction, indicating its effectiveness in helping IPB University students solve technical problems.
Abstract: The IPB Help Center Website is an essential source for IPB University students to get help in solving technical problems. Research was conducted to evaluate the user experience on the IPB Help Center Website through usability testing methods. This research involved 50 IPB University student respondents who used the IPB Help Center website and filled out a survey through Google Forms. The results showed that the overall level of usability was satisfactory, with high ratings for five usability indicators, including learnability, efficiency, memorability, errors, and satisfaction. The average scores of learnability, efficiency, memorability, errors, and satisfaction indicators are 3.88, 3.95, 3.88, 4.40, and 3.85, respectively. This shows that the IPB Help Center Website has been effective in helping users solve technical problems. This research provides an evaluation to improve the user experience and effectiveness of the IPB Help Center website as a valuable resource for IPB University students. Future research will explore the correlation between various usability indicators to gain further insights.
TL;DR: Sure, here is the TLDR: The chapter summarizes traditional phonetics-based approaches to L2 speech and demonstrates the advantage of adopting phonological models to describe and explain the knowledge system of multilinguals.
Abstract: Abstract This chapter provides a summary of traditional, phonetics-based approaches to the study of L2 speech (both production and perception). By contextualizing this approach within the field of language learnability, I demonstrate the advantage of adopting phonological models to describe and explain the knowledge system of multilinguals. Relevant background in the philosophy of mind is provided to justify the modular, rationalist account provided in this book. The role of input (including frequency) is discussed. The chapter ends with a discussion of the levels of the prosodic hierarchy which are part of phonological representations and a recognition that phonology is cognition (not merely physics).
TL;DR: Deep neural networks exhibit a learnability advantage for more structured linguistic input, similar to humans, by systematically generalizing and agreeing with each other, and showing greater similarity to human learners when exposed to compositional languages.
Abstract: Deep neural networks drive the success of natural language processing. A fundamental property of language is its compositional structure, allowing humans to systematically produce forms for new meanings. For humans, languages with more compositional and transparent structures are typically easier to learn than those with opaque and irregular structures. However, this learnability advantage has not yet been shown for deep neural networks, limiting their use as models for human language learning. Here, we directly test how neural networks compare to humans in learning and generalizing different languages that vary in their degree of compositional structure. We evaluate the memorization and generalization capabilities of a large language model and recurrent neural networks, and show that both deep neural networks exhibit a learnability advantage for more structured linguistic input: neural networks exposed to more compositional languages show more systematic generalization, greater agreement between different agents, and greater similarity to human learners.
TL;DR: Researchers propose a new methodology to evaluate the importance of game attributes in simulation games, addressing the lack of standardized methods and interdisciplinary challenges in serious game development and assessment.
Abstract: Information and communication technologies have brought about a revolution in how we perceive, interpret, and learn. This ever-changing landscape presents both challenges and opportunities for enhancing students' learning experiences through innovative tools and strategies. One such strategies is collaboration and serious games into teaching, offering safe environments to solve real-world problems, improve comprehension and knowledge acquisition, develop technology skills, foster cooperation and collaboration, and provide an engaging and interactive learning experience. However, the development of effective simulation games requires careful consideration of various factors, including aesthetics, interface, gameplay, learnability, feedback, challenge, immersion, and game design, to name just a few. Evaluating simulation games poses a challenge due to interdisciplinary involvement and the lack of standardized methodology. This article proposes a methodological approach to assess the attributes of simulation games, providing valuable insights for informed decision-making regarding their design, improvement, effectiveness, and challenges in the realm of serious games.
TL;DR: Predicting quantum learnability from landscape fluctuation is a method to assess the learnability of parametrized quantum circuits regarding a given target. It unifies the effects of insufficient expressibility, barren plateaus, bad local minima, and overparametrization.
Abstract: The tradeoff between trainability and expressibility is a central challenge faced by today's variational quantum computing. Recent studies indicate that resolving this dilemma necessitates designing specific parametrized quantum circuits (PQC) tailored for specific problems, which urgently needs a general and efficient method to assess the learnability of PQCs regarding a given target. In this Letter, we demonstrate a simple and efficient metric for learnability by comparing the fluctuations of the given training landscape with standard learnable landscapes. This metric shows surprising effectiveness in predicting learnability as it unifies the effects of insufficient expressibility, barren plateaus, bad local minima, and overparametrization. Importantly, it does not require actual training and can be estimated efficiently on classical computers via Clifford sampling. We conduct extensive numerical experiments to validate its effectiveness regarding both physical and random Hamiltonians. We also prove a compact lower bound for the metric in locally scrambled circuits as analytical guidance. Our findings enable efficient predictions of learnability, allowing fast selection of suitable PQCs for a given problem without training, which can improve the efficiency of variational quantum computing especially when access to quantum devices is limited.
TL;DR: Noise-tolerant learnability of shallow quantum circuits from statistics and the cost of quantum pseudorandomness is studied. The work proves the natural robustness of quantum statistical queries for learning quantum processes and provides an efficient way to benchmark various classes of noise from statistics.
Abstract: This work studies the learnability of unknown quantum circuits in the near term. We prove the natural robustness of quantum statistical queries for learning quantum processes and provide an efficient way to benchmark various classes of noise from statistics, which gives us a powerful framework for developing noise-tolerant algorithms. We adapt a learning algorithm for constant-depth quantum circuits to the quantum statistical query setting with a small overhead in the query complexity. We prove average-case lower bounds for learning random quantum circuits of logarithmic and higher depths within diamond distance with statistical queries. Additionally, we show the hardness of the quantum threshold search problem from quantum statistical queries and discuss its implications for the learnability of shallow quantum circuits. Finally, we prove that pseudorandom unitaries (PRUs) cannot be constructed using circuits of constant depth by constructing an efficient distinguisher and proving a new variation of the quantum no-free lunch theorem.
TL;DR: The HQCCNN model achieves high classification accuracy on the MNIST dataset, outperforming other models and demonstrating the potential of hybrid quantum–classical convolutional neural networks in image classification.
Abstract: We design a new hybrid quantum–classical convolutional neural network (HQCCNN) model based on parameter quantum circuits. In this model, we use parameterized quantum circuits (PQCs) to redesign the convolutional layer in classical convolutional neural networks, forming a new quantum convolutional layer to achieve unitary transformation of quantum states, enabling the model to more accurately extract hidden information from images. At the same time, we combine the classical fully connected layer with PQCs to form a new hybrid quantum–classical fully connected layer to further improve the accuracy of classification. Finally, we use the MNIST dataset to test the potential of the HQCCNN. The results indicate that the HQCCNN has good performance in solving classification problems. In binary classification tasks, the classification accuracy of numbers 5 and 7 is as high as 99.71%. In multivariate classification, the accuracy rate also reaches 98.51%. Finally, we compare the performance of the HQCCNN with other models and find that the HQCCNN has better classification performance and convergence speed.
TL;DR: This study investigates the effectiveness of mobile augmented reality (AR) with gamification in enhancing learnability for higher institute students of chemistry, demonstrating improved academic achievement, motivation, and retention of lecture contents over traditional teaching methods.
Abstract: Purpose The current study intends to use green-driven augmented reality (AR) with gamification application to help students at the Higher Institute of Science and Technology (HIST) in Libya to effectively learn general chemistry concepts successfully and with minimum side effects on individuals and the environment. It also aims to shed light on the students’ learnability, neural and psychological mechanisms under the green-driven, AR-oriented learning environment that might affect students’ personality, feelings and moods. For this study, smartphones and smart glasses are employed to design AR-G technology. Design/methodology/approach The sample of this study was divided into two groups: the experimental and the control groups. The experimental group used the AR app, and the control group used 2D pictures. The experiment was in two stages: for the first one, a 3D interactive story game reflecting the classroom and the laboratory was designed in which students feel secure and entertained in learning chemistry concepts. In the second stage, the designed gamification solution developed in Unity AR was assessed to measure its acceptability and environmental effects. Findings This study aimed to investigate mobile AR learning experiences. The researchers designed an AR-based game for general chemistry learning, to investigate its effects on students’ behavior, satisfaction and attention. In addition, it intended to uncover the challenges they faced, their experience, concerns about using and the time spent interacting with AR. This study showed that a postlecture activity of testing with AR games affected the retention of lecture contents over 12 weeks significantly better than the retention of the material taught by traditional teaching methodology. Thus, AR-G technology helped to lower students’ test anxiety and increased the regularity of studying. In this study, a student learned in the environment and was liberated from corporeal and sensory connections with their physical surroundings, which greatly aided in improving their experience and collecting players’ learnability analytics, experience, motivation and well-being via game analytics. However, AR-G technology established a competitive learning environment to increase learning by allowing students to be more involved in the learning process and therefore more motivated, resulting in greater real-world performance. On the other hand, that agrees with the latest studies in neuroeducation indicating that it is difficult to learn without conscious and sustained attention. Noreikis et al . (2019) confirmed in their research that action video games could greatly enhance perceptual ability and improve concentration, resulting in a positive impact on learning effectiveness. Research limitations/implications The study has difficulties such as certain hardware being incompatible with the systems of the user device, such as HMD with mobile, and incompatible games. Although AR is not a new technology, one of its challenges is that instructors and students may not be comfortable using it and may not be convinced of the usefulness of technology. One drawback of the current study was that it was limited to a single first-year chemistry class. If the study had been done across several lengthy semesters, it could have had a more beneficial outcome. Another challenge was the small number of participants of students, and their withdrawal for unexpected medical conditions or psychological distress. The choice of one gaming session in a week could generate biased results. Practical implications This study explores how AR with gamification technology can support learning general chemistry topics and shows that AR improved academic achievement and provided instant feedback. The results indicate that AR technology could be helpful in an academic setting by increasing academic achievement and raising motivation for the students who used AR-G technology. Social implications 50% of the interviewees had positive learning experience of AR referring to AR as an enjoyable learning value they gained. One participant commented that “I believe that the augmented reality would be better compared to long texts, and may be suitable for young learners and I feel it is quite efficient and effective”. More than half of the interviewees too considered augmented reality motivating them, triggering their ambition to search for answers to the questions and enhancing further motivated classroom learning. Two interviewees argued that AR potentially develops fun experiences but not necessarily improve learning. Originality/value To meet the objective, 3D interactive story game that imitates the classroom and the laboratory environment in an ecofriendly, entertaining and exciting manner was designed for students of a chemistry course. The first classroom chemistry syllabus was overall divided into three learning units based on their increasing level of difficulty. For each learning unit, the proposed game will offer three modes of green-driven AR-G smart learning. To keep the target students motivated in order to undertake the gaming activity on a regular basis, various motivational affordances will be systematically embedded within the proposed game strategy that includes points, leaderboards, achievements, badges, levels, story, theme, feedback, clear goals, day-to-day challenges and rewards (Deterding et al . (2011), Stott and Neustaedter (2013). Finally, the designed gamification solution was developed in Unity AR, which was later preliminarily tested to evaluate its acceptability and impact on environmental sustainability.
TL;DR: The typology of consonant harmony lacks prefix-controlled systems due to the difficulty of learning the direction of affixation.
Abstract: A notable cross-linguistic gap exists in the typology of consonant harmony: Stem-controlled and suffix-controlled systems are known, but prefix-controlled consonant harmony remains unattested. To explore this gap, an artificial grammar learning (AGL) study was conducted, in which participants were passively trained and then tested on one of four possible sibilant harmony patterns, differing by direction and morphological locus of control. The effect of target-trigger distance was also tested by varying the stem length from one to four syllables. Statistical analysis found significant main effects of Direction and Target-Trigger Distance, with the progressive group outperforming the regressive one and shorter target-trigger distances yielding better performance than longer ones. Affix-controlled groups also learned the pattern more slowly than stem-controlled groups, although overall performance did not differ between affix-controlled and stem-controlled groups. Overall, the results indicate that the typological lack of prefix-controlled consonant harmony is not due to a lack of learnability, as there was no statistically significant difference in ultimate achievement between the four groups.
TL;DR: This study investigates the learnability problem, examining why students struggle to master learning materials, and proposes the concept of "ability to learn" based on psychological characteristics of lagging students, ensuring successful learning outcomes.
Abstract: Learning ability problem. This problem examines the problems that arise in connection with revealing the reasons why students are behind and unable to master. Psychologists put forward the concept of "ability to learn" based on the psychological characteristics of students who are lagging behind in mastering learning materials.
Paul F. Christiano, Jacob Hilton, Victor Lecomte, Mark Xu
4 Sep 2024
TL;DR: Researchers introduce a formal notion of defendability against backdoors, connecting it to learnability and obfuscation. They show that defendability is determined by VC dimension, and that efficient PAC learnability implies efficient defendability, but not conversely.
Abstract: We introduce a formal notion of defendability against backdoors using a game between an attacker and a defender. In this game, the attacker modifies a function to behave differently on a particular input known as the "trigger", while behaving the same almost everywhere else. The defender then attempts to detect the trigger at evaluation time. If the defender succeeds with high enough probability, then the function class is said to be defendable. The key constraint on the attacker that makes defense possible is that the attacker's strategy must work for a randomly-chosen trigger. Our definition is simple and does not explicitly mention learning, yet we demonstrate that it is closely connected to learnability. In the computationally unbounded setting, we use a voting algorithm of Hanneke et al. (2022) to show that defendability is essentially determined by the VC dimension of the function class, in much the same way as PAC learnability. In the computationally bounded setting, we use a similar argument to show that efficient PAC learnability implies efficient defendability, but not conversely. On the other hand, we use indistinguishability obfuscation to show that the class of polynomial size circuits is not efficiently defendable. Finally, we present polynomial size decision trees as a natural example for which defense is strictly easier than learning. Thus, we identify efficient defendability as a notable intermediate concept in between efficient learnability and obfuscation.
TL;DR: CoRE-learning introduces time-sharing resource scheduling in machine learning theory, enabling resource allocation for supercomputing facilities.
Abstract: This article proposes ‘CoRE-learning’ which introduces the ‘time-sharing’ concept and enables ‘resource scheduling’ in intelligent supercomputing facilities to be considered in machine learning theory for the first time.
Fan Chen, Dylan J. Foster, Yanjun Han, Jian Qian, Alexander Rakhlin, Yuqi Xu
7 Oct 2024
TL;DR: This paper develops a unified framework for lower bound methods in statistical estimation and interactive decision making, introducing a novel complexity measure, decision dimension, to derive new bounds and characterize bandit learnability for structured bandit models.
Abstract: In this paper, we develop a unified framework for lower bound methods in statistical estimation and interactive decision making. Classical lower bound techniques -- such as Fano's inequality, Le Cam's method, and Assouad's lemma -- have been central to the study of minimax risk in statistical estimation, yet they are insufficient for the analysis of methods that collect data in an interactive manner. The recent minimax lower bounds for interactive decision making via the Decision-Estimation Coefficient (DEC) appear to be genuinely different from the classical methods. We propose a unified view of these distinct methodologies through a general algorithmic lower bound method. We further introduce a novel complexity measure, decision dimension, which facilitates the derivation of new lower bounds for interactive decision making. In particular, decision dimension provides a characterization of bandit learnability for any structured bandit model class. Further, we characterize the sample complexity of learning convex model class up to a polynomial gap with the decision dimension, addressing the remaining gap between upper and lower bounds in Foster et al. (2021, 2023).
Ilias Diakonikolas, Daniel Kane, Vasilis Kontonis, Sihan Liu, Nikos Zarifis
10 Jun 2024
TL;DR: Efficiently learning low-degree polynomial threshold functions in the presence of a constant fraction of adversarial corruptions.
Abstract: We study the efficient learnability of low-degree polynomial threshold functions (PTFs) in the presence of a constant fraction of adversarial corruptions. Our main algorithmic result is a polynomial-time PAC learning algorithm for this concept class in the strong contamination model under the Gaussian distribution with error guarantee Od, c(opt1−c), for any desired constant c>0, where opt is the fraction of corruptions. In the strong contamination model, an omniscient adversary can arbitrarily corrupt an opt-fraction of the data points and their labels. This model generalizes the malicious noise model and the adversarial label noise model. Prior to our work, known polynomial-time algorithms in this corruption model (or even in the weaker adversarial label noise model) achieved error Õd(opt1/(d+1)), which deteriorates significantly as a function of the degree d. Our algorithm employs an iterative approach inspired by localization techniques previously used in the context of learning linear threshold functions. Specifically, we use a robust perceptron algorithm to compute a good partial classifier and then iterate on the unclassified points. In order to achieve this, we need to take a set defined by a number of polynomial inequalities and partition it into several well-behaved subsets. To this end, we develop new polynomial decomposition techniques that may be of independent interest.
TL;DR: This work investigates the hardness of learning Boolean functions from label proportions, specifically focusing on the intractability of learning OR functions and parities using CNFs and DNFs, respectively, and provides a separation between constant clause CNFs and halfspaces as hypotheses.
Abstract: In recent years the framework of learning from label proportions (LLP) has been gaining importance in machine learning. In this setting, the training examples are aggregated into subsets or bags and only the average label per bag is available for learning an example-level predictor. This generalizes traditional PAC learning which is the special case of unit-sized bags. The computational learning aspects of LLP were studied in recent works (Saket, NeurIPS'21; Saket, NeurIPS'22) which showed algorithms and hardness for learning halfspaces in the LLP setting. In this work we focus on the intractability of LLP learning Boolean functions. Our first result shows that given a collection of bags of size at most $2$ which are consistent with an OR function, it is NP-hard to find a CNF of constantly many clauses which satisfies any constant-fraction of the bags. This is in contrast with the work of (Saket, NeurIPS'21) which gave a $(2/5)$-approximation for learning ORs using a halfspace. Thus, our result provides a separation between constant clause CNFs and halfspaces as hypotheses for LLP learning ORs. Next, we prove the hardness of satisfying more than $1/2 + o(1)$ fraction of such bags using a $t$-DNF (i.e. DNF where each term has $\leq t$ literals) for any constant $t$. In usual PAC learning such a hardness was known (Khot-Saket, FOCS'08) only for learning noisy ORs. We also study the learnability of parities and show that it is NP-hard to satisfy more than $(q/2^{q-1} + o(1))$-fraction of $q$-sized bags which are consistent with a parity using a parity, while a random parity based algorithm achieves a $(1/2^{q-2})$-approximation.