Exploring exploratory programming

TL;DR: The Gamma as mentioned in this paper is a text-based data exploration environment that allows non-experts to access any data source, result in reproducible scripts and encourage users to verify, reuse and modify existing code.

TL;DR: This paper proposes HAConvGNN, a hierarchical attention-based model for code documentation generation in Jupyter Notebooks, outperforming baselines on a new corpus of well-documented Kaggle notebooks by considering relevant code cells and tokens.

Abstract: Data scientists often formulate predictive modeling tasks involving fuzzy, hard-to-define concepts, such as the ''authenticity'' of student writing or the ''healthcare need'' of a patient. Yet the process by which data scientists translate fuzzy concepts into a concrete, proxy target variable remains poorly understood. We interview fifteen data scientists in education (N=8) and healthcare (N=7) to understand how they construct target variables for predictive modeling tasks. Our findings suggest that data scientists construct target variables through a bricolage process, in which they use creative and pragmatic approaches to make do with the limited data at hand. Data scientists attempt to satisfy five major criteria for a target variable through bricolage: validity, simplicity, predictability, portability, and resource requirements. To achieve this, data scientists adaptively apply problem (re)formulation strategies, such as swapping out one candidate target variable for another when the first fails to meet certain criteria (e.g., predictability), or composing multiple outcomes into a single target variable to capture a more holistic set of modeling objectives. Based on our findings, we present opportunities for future HCI, CSCW, and ML research to better support the art and science of target variable construction.

TL;DR: Forking as mentioned in this paper is an extension to notebook that helps data scientists more directly express and navigate through decision points in a single notebook, which is useful for exploring alternatives and debugging code to create a predictive model.

TL;DR: This work describes new algorithms that take into account the variable cost of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation and shows that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms.

TL;DR: A description of some typical EDA procedures is given and some of the principles of EDA are discussed.

TL;DR: "Digital fluency" should mean designing, creating, and remixing, not just browsing, chatting, and interacting.