Free parameter

Abstract: We introduce a parametrized high-density equation of state (EOS) in order to systematize the study of constraints placed by astrophysical observations on the nature of neutron-star matter To obtain useful constraints, the number of parameters must be smaller than the number of EOS-related neutron-star properties measured, but large enough to accurately approximate the large set of candidate EOSs We find that a parametrized EOS based on piecewise polytropes with 3 free parameters matches, to about 4% rms error, an extensive set of candidate EOSs at densities below the central density of $14{M}_{\ensuremath{\bigodot}}$ stars Adding observations of more massive stars constrains the higher-density part of the EOS and requires an additional parameter We obtain constraints on the allowed parameter space set by causality and by present and near-future astronomical observations with the least model dependence Stringent constraints on the EOS parameter space are associated with the future measurement of the moment of inertia of PSR J0737-3039A combined with the maximum known neutron-star mass We also present in an appendix a more efficient algorithm than has previously been used for finding points of marginal stability and the maximum angular velocity of stable stars

Abstract: Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a {\em biased} estimate of the gradient of the {\em average reward} in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter $\beta\in [0,1)$ (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter $\beta$ is related to the {\em mixing time} of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward

Abstract: Conditional maximum entropy (ME) models provide a general purpose machine learning technique which has been successfully applied to fields as diverse as computer vision and econometrics, and which is used for a wide variety of classification problems in natural language processing. However, the flexibility of ME models is not without cost. While parameter estimation for ME models is conceptually straightforward, in practice ME models for typical natural language tasks are very large, and may well contain many thousands of free parameters. In this paper, we consider a number of algorithms for estimating the parameters of ME models, including iterative scaling, gradient ascent, conjugate gradient, and variable metric methods. Sur-prisingly, the standardly used iterative scaling algorithms perform quite poorly in comparison to the others, and for all of the test problems, a limited-memory variable metric algorithm outperformed the other choices.