Consistency (knowledge bases)

Abstract: Three assumptions sufficient to identify the average causal effect are consistency, positivity, and exchangeability (ie, “no unmeasured confounders and no informative censoring,” or “ignorability of the treatment assignment and measurement of the outcome”). The exchangeability assumptions are well known territory for epidemiologists and biostatisticians. Briefly, to be satisfied, these 2 exchangeability assumptions that require exposed and unexposed subjects, and censored and uncensored subjects have equal distributions of potential outcomes, respectively. Indeed, the so-called fundamental problem of causal inference is directly linked to the first exchangeability assumption. In contrast, the consistency and positivity assumptions are less well known. The positivity assumption states that there is a nonzero (ie, positive) probability of receiving every level of exposure for every combination of values of exposure and confounders that occur among individuals in the population. It remains unclear why the consistency and positivity assumptions are less well known. Optimistically, perhaps these assumptions are less important with respect to an impact on estimation of the average causal effect. Pessimistically, these assumptions are less well known because there is little alarming evidence of a departure from either of these assumptions in observational studies without explicitly looking for the departure. Here we will focus on the preliminary issue of clarifying the consistency assumption. The consistency assumption is often stated such that an individual’s potential outcome under her observed exposure history is precisely her observed outcome. Methods for causal inference require that the exposure is defined unambiguously. Specifically, one needs to be able to explain how a certain level of exposure could be hypothetically assigned to a person exposed to a different level. This requirement is known as consistency. Consistency is guaranteed by design in experiments, because application of the exposure to any individual is under the control of the investigator. Consistency is plausible in observational studies of medical treatments, because one can imagine how to manipulate hypothetically an individual’s treatment status. However, consistency is problematic in observational studies with exposures for which manipulation is difficult to conceive. Consistency is especially difficult when the exposure is a biologic feature, such as body weight, insulin resistance, or CD4 cell count. For example, there are many competing ways to assign (hypothetically) a body mass index of 25 kg/m to an individual, and each of them may have a different causal effect on the outcome. To state consistency formally, let us first define individual j’s potential outcome Yj(x) under exposure x as the outcome that would have been observed if individual j had received exposure x. The variable Yj(x) is known as a potential outcome because it

Abstract: Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%).

Abstract: Cole and Frangakis (Epidemiology. 2009;20:3-5) introduced notation for the consistency assumption in causal inference. I extend this notation and propose a refinement of the consistency assumption that makes clear that the consistency statement, as ordinarily given, is in fact an assumption and not an axiom or definition. The refinement is also useful in showing that additional assumptions (referred to here as treatment-variation irrelevance assumptions), stronger than those given by Cole and Frangakis, are in fact necessary in articulating the ordinary assumptions of ignorability or exchangeability. The refinement furthermore sheds light on the distinction between intervention and choice in reasoning about causality. A distinction between the range of treatment variations for which potential outcomes can be defined and the range for which treatment comparisons are made is discussed in relation to issues of nonadherence. The use of stochastic counterfactuals can help relax what is effectively being presupposed by the treatment-variation irrelevance assumption and the consistency assumption.

Abstract: Deep semi-supervised learning is a fast-growing field with a range of practical applications. This paper provides a comprehensive survey on both fundamentals and recent advances in deep semi-supervised learning methods from perspectives of model design and unsupervised loss functions. We first present a taxonomy for deep semi-supervised learning that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods. Then we provide a comprehensive review of 60 representative methods and offer a detailed comparison of these methods in terms of the type of losses, architecture differences, and test performance results. In addition to the progress in the past few years, we further discuss some shortcomings of existing methods and provide some tentative heuristic solutions for solving these open problems.