Statistics
New submissions
[ showing up to 2000 entries per page: fewer | more ]
New submissions for Fri, 1 Jan 21
- [1] arXiv:2012.14482 [pdf, other]
-
Title: Multivariate Smoothing via the Fourier Integral Theorem and Fourier KernelComments: 58 pages, 6 figuresSubjects: Statistics Theory (math.ST); Methodology (stat.ME); Machine Learning (stat.ML)
Starting with the Fourier integral theorem, we present natural Monte Carlo estimators of multivariate functions including densities, mixing densities, transition densities, regression functions, and the search for modes of multivariate density functions (modal regression). Rates of convergence are established and, in many cases, provide superior rates to current standard estimators such as those based on kernels, including kernel density estimators and kernel regression functions. Numerical illustrations are presented.
- [2] arXiv:2012.14530 [pdf, ps, other]
-
Title: On the T-testAuthors: S.Y. NovakComments: 19 pagesSubjects: Statistics Theory (math.ST); Methodology (stat.ME)
The $T$-test is probably the most popular statistical test; it is routinely recommended by the textbooks. The applicability of the test relies upon the validity of normal or Student's approximation to the distribution of Student's statistic $\,t_n$. However, the latter assumption is not valid as often as assumed. We show that normal or Student's approximation to $\,\L(t_n)\,$ does not hold uniformly even in the class $\,{\cal P}_n\,$ of samples from zero-mean unit-variance bounded distributions. We present lower bounds to the corresponding error. The fact that a non-parametric test is not applicable uniformly to samples from the class $\,{\cal P}_n\,$ seems to be established for the first time. It means the $T$-test can be misleading, and should not be recommended in its present form. We suggest a generalisation of the test that allows for variability of possible limiting/approximating distributions to $\,\L(t_n)$.
- [3] arXiv:2012.14563 [pdf, other]
-
Title: Random Planted Forest: a directly interpretable tree ensembleSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
We introduce a novel interpretable and tree-based algorithm for prediction in a regression setting in which each tree in a classical random forest is replaced by a family of planted trees that grow simultaneously. The motivation for our algorithm is to estimate the unknown regression function from a functional ANOVA decomposition perspective, where each tree corresponds to a function within that decomposition. Therefore, planted trees are limited in the number of interaction terms. The maximal order of approximation in the ANOVA decomposition can be specified or left unlimited. If a first order approximation is chosen, the result is an additive model. In the other extreme case, if the order of approximation is not limited, the resulting model puts no restrictions on the form of the regression function. In a simulation study we find encouraging prediction and visualisation properties of our random planted forest method. We also develop theory for an idealised version of random planted forests in the case of an underlying additive model. We show that in the additive case, the idealised version achieves up to a logarithmic factor asymptotically optimal one-dimensional convergence rates of order $n^{-2/5}$.
- [4] arXiv:2012.14589 [pdf, other]
-
Title: Sample Size Re-estimation Design in Phase II Dose Finding Study with Multiple Dose Groups: Frequentist and Bayesian MethodsSubjects: Methodology (stat.ME); Applications (stat.AP)
Unblinded sample size re-estimation (SSR) is often planned in a clinical trial when there is large uncertainty about the true treatment effect. For Proof-of Concept (PoC) in a Phase II dose finding study, contrast test can be adopted to leverage information from all treatment groups. In this article, we propose two-stage SSR designs using frequentist conditional power and Bayesian posterior predictive power for both single and multiple contrast tests. The Bayesian SSR can be implemented under a wide range of prior settings to incorporate different prior knowledge. Taking the adaptivity into account, all type I errors of final analysis in this paper are rigorously protected. Simulation studies are carried out to demonstrate the advantages of unblinded SSR in multi-arm trials.
- [5] arXiv:2012.14614 [pdf, other]
-
Title: A Survey of Parameter and State Estimation in QueuesSubjects: Methodology (stat.ME); Probability (math.PR)
We present a broad literature survey of parameter and state estimation for queueing systems. Our approach is based on various inference activities, queueing models, observations schemes, and statistical methods. We categorize these into branches of research that we call estimation paradigms. These include: the classical sampling approach, inverse problems, inference for non-interacting systems, inference with discrete sampling, inference with queueing fundamentals, queue inference engine problems, Bayesian approaches, online prediction, implicit models, and control, design, and uncertainty quantification. For each of these estimation paradigms, we outline the principles and ideas, while surveying key references. We also present various simple numerical experiments. In addition to some key references mentioned here, a periodically-updated comprehensive list of references dealing with parameter and state estimation of queues will be kept in an accompanying annotated bibliography.
- [6] arXiv:2012.14657 [pdf, other]
-
Title: Behavior of linear L2-boosting algorithms in the vanishing learning rate asymptoticComments: 36 pagesSubjects: Machine Learning (stat.ML); Probability (math.PR); Statistics Theory (math.ST)
We investigate the asymptotic behaviour of gradient boosting algorithms when the learning rate converges to zero and the number of iterations is rescaled accordingly. We mostly consider L2-boosting for regression with linear base learner as studied in B{\"u}hlmann and Yu (2003) and analyze also a stochastic version of the model where subsampling is used at each step (Friedman 2002). We prove a deterministic limit in the vanishing learning rate asymptotic and characterize the limit as the unique solution of a linear differential equation in an infinite dimensional function space. Besides, the training and test error of the limiting procedure are thoroughly analyzed. We finally illustrate and discuss our result on a simple numerical experiment where the linear L2-boosting operator is interpreted as a smoothed projection and time is related to its number of degrees of freedom.
- [7] arXiv:2012.14708 [pdf, ps, other]
-
Title: Adaptive Estimation for Non-stationary Factor Models And A Test for Static Factor LoadingsSubjects: Methodology (stat.ME)
This paper considers the estimation and testing of a class of high-dimensional non-stationary time series factor models with evolutionary temporal dynamics. In particular, the entries and the dimension of the factor loading matrix are allowed to vary with time while the factors and the idiosyncratic noise components are locally stationary. We propose an adaptive sieve estimator for the span of the varying loading matrix and the locally stationary factor processes. A uniformly consistent estimator of the effective number of factors is investigated via eigenanalysis of a non-negative definite time-varying matrix. A high-dimensional bootstrap-assisted test for the hypothesis of static factor loadings is proposed by comparing the kernels of the covariance matrices of the whole time series with their local counterparts. We examine our estimator and test via simulation studies and a real data analysis.
- [8] arXiv:2012.14759 [pdf, other]
-
Title: New statistical control limits using maximum copula entropySubjects: Applications (stat.AP)
Statistical quality control methods are noteworthy to produced standard production in manufacturing processes. In this regard, there are many classical manners to control the process. Many of them have a global assumption around distributions of the process data. They are supposed to be normal, which is clear that it is not always valid for all processes. Such control charts made some false decisions that waste funds. So, the main question while working with multivariate data set is how to find the multivariate distribution of the data set, which saves the original dependency between variables. Up to our knowledge, a copula function guarantees the dependence on the result function. But it is not enough when there is no other functional information about the statistical society, and we have just a data set. Therefore, we apply the maximum entropy concept to deal with this situation. In this paper, first of all, we find out the joint distribution of a data set, which is from a manufacturing process that needs to be control while running the production process. Then, we get an elliptical control limit via the maximum copula entropy. In the final step, we represent a practical example using the stated method. Average run lengths are calculated for some means and shifts to show the ability of the maximum copula entropy. In the end, two real data examples are presented.
- [9] arXiv:2012.14804 [pdf, other]
-
Title: Kernel Partial Correlation Coefficient -- a Measure of Conditional DependenceComments: 63 pages, 4 figures, 9 tablesSubjects: Methodology (stat.ME)
In this paper we propose and study a class of simple, nonparametric, yet interpretable measures of conditional dependence between two random variables $Y$ and $Z$ given a third variable $X$, all taking values in general topological spaces. The population version of any of these measures captures the strength of conditional dependence and it is 0 if and only if $Y$ and $Z$ are conditionally independent given $X$, and 1 if and only if $Y$ is a measurable function of $Z$ and $X$. Thus, our measure -- which we call kernel partial correlation (KPC) coefficient -- can be thought of as a nonparametric generalization of the partial correlation coefficient that possesses the above properties when $(X,Y,Z)$ is jointly normal. We describe two consistent methods of estimating KPC. Our first method utilizes the general framework of geometric graphs, including $K$-nearest neighbor graphs and minimum spanning trees. A sub-class of these estimators can be computed in near linear time and converges at a rate that automatically adapts to the intrinsic dimension(s) of the underlying distribution(s). Our second strategy involves direct estimation of conditional mean embeddings using cross-covariance operators in the reproducing kernel Hilbert spaces. Using these empirical measures we develop forward stepwise (high-dimensional) nonlinear variable selection algorithms. We show that our algorithm, using the graph-based estimator, yields a provably consistent model-free variable selection procedure, even in the high-dimensional regime when the number of covariates grows exponentially with the sample size, under suitable sparsity assumptions. Extensive simulation and real-data examples illustrate the superior performance of our methods compared to existing procedures. The recent conditional dependence measure proposed by Azadkia and Chatterjee (2019) can be viewed as a special case of our general framework.
- [10] arXiv:2012.14817 [pdf, other]
-
Title: Spatial Resolution Enhancement of Oversampled Images Using Regression Decomposition and SynthesisAuthors: Hsien-Wei ChenSubjects: Applications (stat.AP); Methodology (stat.ME)
A new statistical model designed for regression analysis with a sparse design matrix is proposed. This new model utilizes the positions of the limited non-zero elements in the design matrix to decompose the regression model into sub-regression models. Statistical inferences are further made on the values of these limited non-zero elements to provide a reference for synthesizing these sub-regression models. With this concept of the regression decomposition and synthesis, the information on the structure of the design matrix can be incorporated into the regression analysis to provide a more reliable estimation. The proposed model is then applied to resolve the spatial resolution enhancement problem for spatially oversampled images. To systematically evaluate the performance of the proposed model in enhancing the spatial resolution, the proposed approach is applied to the oversampled images that are reproduced via random field simulations. These application results based on different generated scenarios then conclude the effectiveness and the feasibility of the proposed approach in enhancing the spatial resolution of spatially oversampled images.
- [11] arXiv:2012.14844 [pdf, other]
-
Title: Inference for Low-rank Tensors -- No Need to DebiasSubjects: Statistics Theory (math.ST); Machine Learning (cs.LG); Methodology (stat.ME); Machine Learning (stat.ML)
In this paper, we consider the statistical inference for several low-rank tensor models. Specifically, in the Tucker low-rank tensor PCA or regression model, provided with any estimates achieving some attainable error rate, we develop the data-driven confidence regions for the singular subspace of the parameter tensor based on the asymptotic distribution of an updated estimate by two-iteration alternating minimization. The asymptotic distributions are established under some essential conditions on the signal-to-noise ratio (in PCA model) or sample size (in regression model). If the parameter tensor is further orthogonally decomposable, we develop the methods and theory for inference on each individual singular vector. For the rank-one tensor PCA model, we establish the asymptotic distribution for general linear forms of principal components and confidence interval for each entry of the parameter tensor. Finally, numerical simulations are presented to corroborate our theoretical discoveries.
In all these models, we observe that different from many matrix/vector settings in existing work, debiasing is not required to establish the asymptotic distribution of estimates or to make statistical inference on low-rank tensors. In fact, due to the widely observed statistical-computational-gap for low-rank tensor estimation, one usually requires stronger conditions than the statistical (or information-theoretic) limit to ensure the computationally feasible estimation is achievable. Surprisingly, such conditions ``incidentally" render a feasible low-rank tensor inference without debiasing. - [12] arXiv:2012.14847 [pdf, other]
-
Title: Scalable Multivariate HistogramsSubjects: Computation (stat.CO); Distributed, Parallel, and Cluster Computing (cs.DC); Optimization and Control (math.OC)
We give a distributed variant of an adaptive histogram estimation procedure previously developed by the first author. The procedure is based on regular pavings and is known to have numerous appealing statistical and arithmetical properties. The distributed version makes it possible to process data sets significantly bigger than previously. We provide prototype implementation under a permissive license.
- [13] arXiv:2012.14881 [pdf, ps, other]
-
Title: A general perspective on the Metropolis-Hastings kernelSubjects: Computation (stat.CO); Methodology (stat.ME)
Since its inception the Metropolis-Hastings kernel has been applied in sophisticated ways to address ever more challenging and diverse sampling problems. Its success stems from the flexibility brought by the fact that its verification and sampling implementation rests on a local ``detailed balance'' condition, as opposed to a global condition in the form of a typically intractable integral equation. While checking the local condition is routine in the simplest scenarios, this proves much more difficult for complicated applications involving auxiliary structures and variables. Our aim is to develop a framework making establishing correctness of complex Markov chain Monte Carlo kernels a purely mechanical or algebraic exercise, while making communication of ideas simpler and unambiguous by allowing a stronger focus on essential features -- a choice of embedding distribution, an involution and occasionally an acceptance function -- rather than the induced, boilerplate structure of the kernels that often tends to obscure what is important. This framework can also be used to validate kernels that do not satisfy detailed balance, i.e. which are not reversible, but a modified version thereof.
- [14] arXiv:2012.14894 [pdf, other]
-
Title: Statistical Formulas for F MeasuresAuthors: Wenxin JiangSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
We provide analytic formulas for the standard error and confidence intervals for the F measures, based on a property of asymptotic normality in the large sample limit. The formula can be applied for sample size planning in order to achieve accurate enough estimation of these F measures.
- [15] arXiv:2012.14932 [pdf, other]
-
Title: An extension of the angular synchronization problem to the heterogeneous settingComments: 45 pages, 17 figuresSubjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Numerical Analysis (math.NA)
Given an undirected measurement graph $G = ([n], E)$, the classical angular synchronization problem consists of recovering unknown angles $\theta_1,\dots,\theta_n$ from a collection of noisy pairwise measurements of the form $(\theta_i - \theta_j) \mod 2\pi$, for each $\{i,j\} \in E$. This problem arises in a variety of applications, including computer vision, time synchronization of distributed networks, and ranking from preference relationships. In this paper, we consider a generalization to the setting where there exist $k$ unknown groups of angles $\theta_{l,1}, \dots,\theta_{l,n}$, for $l=1,\dots,k$. For each $ \{i,j\} \in E$, we are given noisy pairwise measurements of the form $\theta_{\ell,i} - \theta_{\ell,j}$ for an unknown $\ell \in \{1,2,\ldots,k\}$. This can be thought of as a natural extension of the angular synchronization problem to the heterogeneous setting of multiple groups of angles, where the measurement graph has an unknown edge-disjoint decomposition $G = G_1 \cup G_2 \ldots \cup G_k$, where the $G_i$'s denote the subgraphs of edges corresponding to each group. We propose a probabilistic generative model for this problem, along with a spectral algorithm for which we provide a detailed theoretical analysis in terms of robustness against both sampling sparsity and noise. The theoretical findings are complemented by a comprehensive set of numerical experiments, showcasing the efficacy of our algorithm under various parameter regimes. Finally, we consider an application of bi-synchronization to the graph realization problem, and provide along the way an iterative graph disentangling procedure that uncovers the subgraphs $G_i$, $i=1,\ldots,k$ which is of independent interest, as it is shown to improve the final recovery accuracy across all the experiments considered.
- [16] arXiv:2012.14936 [pdf, other]
-
Title: Learning Energy-Based Model with Variational Auto-Encoder as Amortized SamplerSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Due to the intractable partition function, training energy-based models (EBMs) by maximum likelihood requires Markov chain Monte Carlo (MCMC) sampling to approximate the gradient of the Kullback-Leibler divergence between data and model distributions. However, it is non-trivial to sample from an EBM because of the difficulty of mixing between modes. In this paper, we propose to learn a variational auto-encoder (VAE) to initialize the finite-step MCMC, such as Langevin dynamics that is derived from the energy function, for efficient amortized sampling of the EBM. With these amortized MCMC samples, the EBM can be trained by maximum likelihood, which follows an "analysis by synthesis" scheme; while the variational auto-encoder learns from these MCMC samples via variational Bayes. We call this joint training algorithm the variational MCMC teaching, in which the VAE chases the EBM toward data distribution. We interpret the learning algorithm as a dynamic alternating projection in the context of information geometry. Our proposed models can generate samples comparable to GANs and EBMs. Additionally, we demonstrate that our models can learn effective probabilistic distribution toward supervised conditional learning experiments.
- [17] arXiv:2012.14944 [pdf, other]
-
Title: Learning non-stationary Langevin dynamics from stochastic observations of latent trajectoriesSubjects: Machine Learning (stat.ML); Statistical Mechanics (cond-mat.stat-mech); Machine Learning (cs.LG); Biological Physics (physics.bio-ph); Data Analysis, Statistics and Probability (physics.data-an)
Many complex systems operating far from the equilibrium exhibit stochastic dynamics that can be described by a Langevin equation. Inferring Langevin equations from data can reveal how transient dynamics of such systems give rise to their function. However, dynamics are often inaccessible directly and can be only gleaned through a stochastic observation process, which makes the inference challenging. Here we present a non-parametric framework for inferring the Langevin equation, which explicitly models the stochastic observation process and non-stationary latent dynamics. The framework accounts for the non-equilibrium initial and final states of the observed system and for the possibility that the system's dynamics define the duration of observations. Omitting any of these non-stationary components results in incorrect inference, in which erroneous features arise in the dynamics due to non-stationary data distribution. We illustrate the framework using models of neural dynamics underlying decision making in the brain.
- [18] arXiv:2012.14949 [pdf, other]
-
Title: Estimating the change in soccer's home advantage during the Covid-19 pandemic using bivariate Poisson regressionSubjects: Applications (stat.AP)
In wake of the Covid-19 pandemic, 2019-2020 soccer seasons across the world were postponed and eventually made up during the summer months of 2020. Researchers from a variety of disciplines jumped at the opportunity to compare the rescheduled games, played in front of empty stadia, to previous games, played in front of fans. To date, most of this post-Covid soccer research has used linear regression models, or versions thereof, to estimate potential changes to the home advantage. But because soccer outcomes are non-linear, we argue that leveraging the Poisson distribution would be more appropriate. We begin by using simulations to show that bivariate Poisson regression reduces absolute bias when estimating the home advantage benefit in a single season of soccer games, relative to linear regression, by almost 85 percent. Next, with data from 17 professional soccer leagues, we extend bivariate Poisson models estimate the change in home advantage due to games being played without fans. In contrast to current research that overwhelmingly suggests a drop in the home advantage, our findings are mixed; in some leagues, evidence points to a decrease, while in others, the home advantage may have risen. Altogether, this suggests a more complex causal mechanism for the impact of fans on sporting events.
- [19] arXiv:2012.14951 [pdf, other]
-
Title: Bridging Cost-sensitive and Neyman-Pearson Paradigms for Asymmetric Binary ClassificationSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Asymmetric binary classification problems, in which the type I and II errors have unequal severity, are ubiquitous in real-world applications. To handle such asymmetry, researchers have developed the cost-sensitive and Neyman-Pearson paradigms for training classifiers to control the more severe type of classification error, say the type I error. The cost-sensitive paradigm is widely used and has straightforward implementations that do not require sample splitting; however, it demands an explicit specification of the costs of the type I and II errors, and an open question is what specification can guarantee a high-probability control on the population type I error. In contrast, the Neyman-Pearson paradigm can train classifiers to achieve a high-probability control of the population type I error, but it relies on sample splitting that reduces the effective training sample size. Since the two paradigms have complementary strengths, it is reasonable to combine their strengths for classifier construction. In this work, we for the first time study the methodological connections between the two paradigms, and we develop the TUBE-CS algorithm to bridge the two paradigms from the perspective of controlling the population type I error.
- [20] arXiv:2012.15047 [pdf, other]
-
Title: Adjusted chi-square test for degree-corrected block modelsSubjects: Statistics Theory (math.ST); Social and Information Networks (cs.SI); Machine Learning (stat.ML)
We propose a goodness-of-fit test for degree-corrected stochastic block models (DCSBM). The test is based on an adjusted chi-square statistic for measuring equality of means among groups of $n$ multinomial distributions with $d_1,\dots,d_n$ observations. In the context of network models, the number of multinomials, $n$, grows much faster than the number of observations, $d_i$, hence the setting deviates from classical asymptotics. We show that a simple adjustment allows the statistic to converge in distribution, under null, as long as the harmonic mean of $\{d_i\}$ grows to infinity. This result applies to large sparse networks where the role of $d_i$ is played by the degree of node $i$. Our distributional results are nonasymptotic, with explicit constants, providing finite-sample bounds on the Kolmogorov-Smirnov distance to the target distribution. When applied sequentially, the test can also be used to determine the number of communities. The test operates on a (row) compressed version of the adjacency matrix, conditional on the degrees, and as a result is highly scalable to large sparse networks. We incorporate a novel idea of compressing the columns based on a $(K+1)$-community assignment when testing for $K$ communities. This approach increases the power in sequential applications without sacrificing computational efficiency, and we prove its consistency in recovering the number of communities. Since the test statistic does not rely on a specific alternative, its utility goes beyond sequential testing and can be used to simultaneously test against a wide range of alternatives outside the DCSBM family. We show the effectiveness of the approach by extensive numerical experiments with simulated and real data. In particular, applying the test to the Facebook-100 dataset, we find that a DCSBM with a small number of communities is far from a good fit in almost all cases.
- [21] arXiv:2012.15093 [pdf, other]
-
Title: Closed test procedures for the comparison of dose groups against a negative control group or placeboAuthors: Ludwig A. HothornComments: 1 figure, 1 tableSubjects: Applications (stat.AP)
Dose groups are compared with a control assuming an order restriction usually by the Williams trend test. Here, as an alternative, two variants of the closed testing procedure are considered, one where global Williams tests are used in the partition hypotheses, and another where pairwise contrast tests are used for this purpose. Related R software is provided.
- [22] arXiv:2012.15130 [pdf, other]
-
Title: Spatio-temporal methods for estimating subsurface ocean thermal response to tropical cyclonesComments: 32 pages, 14 figures, supplement and code at this https URLSubjects: Applications (stat.AP)
Prevailing theory contends that tropical cyclones (TCs) are primarily driven by wind-induced heat exchange between the air and sea. Accurate characterization of the subsurface ocean thermal response to TC passage is crucial for accurate TC intensity forecasts and climatological modeling, yet that characterization is complicated by the high-noise ocean environment, correlations inherent in spatio-temporal data, relative scarcity of in situ observations, and entanglement of the TC-induced signal with seasonal signals. We present a general methodological framework that addresses these difficulties, integrating existing techniques in seasonal mean field estimation, Gaussian process modeling, and nonparametric regression into a functional ANOVA model. Importantly, we improve upon past work by properly handling seasonality, providing rigorous uncertainty quantification, and treating time as a continuous variable, rather than producing estimates that are binned in time. This functional ANOVA model is estimated using in situ subsurface temperature profiles from the Argo fleet of autonomous floats through a multi-step procedure, which (1) characterizes the upper ocean seasonal shift during the TC season; (2) models the variability in the temperature observations; (3) fits a thin plate spline using the variability estimates to account for heteroskedasticity and correlation between the observations. This spline fit reveals the ocean thermal response to TC passage. Through this framework, we obtain new scientific insights into the interaction between TCs and the ocean on a global scale, including a three-dimensional characterization of the near-surface and subsurface cooling along the TC storm track and the mixing-induced subsurface warming on the track's right side.
- [23] arXiv:2012.15274 [pdf, other]
-
Title: Provably Training Neural Network Classifiers under Fairness ConstraintsSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
Training a classifier under fairness constraints has gotten increasing attention in the machine learning community thanks to moral, legal, and business reasons. However, several recent works addressing algorithmic fairness have only focused on simple models such as logistic regression or support vector machines due to non-convex and non-differentiable fairness criteria across protected groups, such as race or gender. Neural networks, the most widely used models for classification nowadays, are precluded and lack theoretical guarantees. This paper aims to fill this missing but crucial part of the literature of algorithmic fairness for neural networks. In particular, we show that overparametrized neural networks could meet the fairness constraints. The key ingredient of building a fair neural network classifier is establishing no-regret analysis for neural networks in the overparameterization regime, which may be of independent interest in the online learning of neural networks and related applications.
- [24] arXiv:2012.15278 [pdf, other]
-
Title: An automatic procedure to determine groups of nonparametric regression curvesSubjects: Methodology (stat.ME); Machine Learning (stat.ML)
In many situations it could be interesting to ascertain whether nonparametric regression curves can be grouped, especially when confronted with a considerable number of curves. The proposed testing procedure allows to determine groups with an automatic selection of their number. A simulation study is presented in order to investigate the finite sample properties of the proposed methods when compared to existing alternative procedures. Finally, the applicability of the procedure to study the geometry of a tunnel by analysing a set of cross-sections is demonstrated. The results obtained show the existence of some heterogeneity in the tunnel geometry.
- [25] arXiv:2012.15301 [pdf, other]
-
Title: Optimal trees selection for classification via out-of-bag assessment and sub-baggingComments: 20 pages,4 figuresSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
The effect of training data size on machine learning methods has been well investigated over the past two decades. The predictive performance of tree based machine learning methods, in general, improves with a decreasing rate as the size of training data increases. We investigate this in optimal trees ensemble (OTE) where the method fails to learn from some of the training observations due to internal validation. Modified tree selection methods are thus proposed for OTE to cater for the loss of training observations in internal validation. In the first method, corresponding out-of-bag (OOB) observations are used in both individual and collective performance assessment for each tree. Trees are ranked based on their individual performance on the OOB observations. A certain number of top ranked trees is selected and starting from the most accurate tree, subsequent trees are added one by one and their impact is recorded by using the OOB observations left out from the bootstrap sample taken for the tree being added. A tree is selected if it improves predictive accuracy of the ensemble. In the second approach, trees are grown on random subsets, taken without replacement-known as sub-bagging, of the training data instead of bootstrap samples (taken with replacement). The remaining observations from each sample are used in both individual and collective assessments for each corresponding tree similar to the first method. Analysis on 21 benchmark datasets and simulations studies show improved performance of the modified methods in comparison to OTE and other state-of-the-art methods.
- [26] arXiv:2012.15302 [pdf]
-
Title: An analysis to identify the structural breaks of COVID-19 in TurkeyComments: 23 pages, 12 figures, 4 tablesSubjects: Applications (stat.AP)
In countries with a severe outbreak of COVID-19, most governments are considering whether anti-transmission measures are worth social and economic costs. The seriousness of economic costs such as the closure of some workplaces, unemployment, reduction in production, and social costs such as school closures, disruptions in education could be observable. However, the effect of the measures taken on the spread of the epidemic, such as the number of delayed or prevented cases, could not be observed. For this reason, the direct effects of the measures taken on health, that is, the effects on the course of the epidemic, are important research subjects. For this purpose, in this study, the breakpoint linear regression analysis is performed to analyze the trends of daily active cases, recovered, and deaths in Turkey. The analysis reveals that there has been a remarkable impact on lockdown and other precautions. Using the breakpoint regression model, we also analyze the active cases' trajectory for eight affected countries and compare the patterns in these countries with Turkey.
- [27] arXiv:2012.15306 [pdf, other]
-
Title: Bayesian state space modelling for COVID-19: with Tennessee and New York case studiesSubjects: Methodology (stat.ME); Applications (stat.AP)
We develop a Bayesian inferential framework for the spread of COVID-19 using mechanistic epidemiological models, such as SIR or SEIR, and allow the effective contact rate to vary in time. A novel aspect of our approach is the incorporation of a time-varying reporting rate accounting for the initial phase of the pandemic before testing was widely available. By varying both the reporting rate and the effective contact rate in time, our models can capture changes in the data induced by external influences, such as public health intervention measures, for example. We view COVID-19 incidence data as the observed measurements of a hidden Markov model, with latent space represented by the underlying epidemiological model, and employ a particle Markov chain Monte Carlo (PMCMC) sampling scheme for Bayesian inference. Parameter inference is performed via PMCMC on incidence data collated by the New York Times from the states of New York and Tennessee from March 1, 2020 through August 30, 2020. Lastly, we perform Bayesian model selection on the different formulations of the epidemiological models, make predictions from our fitted models, and validate our predictions against the true incidence data for the week between August 31, 2020 and September 7, 2020.
- [28] arXiv:2012.15313 [pdf, other]
-
Title: Learning Sparsity and Block Diagonal Structure in Multi-View Mixture ModelsAuthors: Iain CarmichaelComments: First upload, 39 pages and 15 figuresSubjects: Methodology (stat.ME)
Scientific studies increasingly collect multiple modalities of data to investigate a phenomenon from several perspectives. In integrative data analysis it is important to understand how information is heterogeneously spread across these different data sources. To this end, we consider a parametric clustering model for the subjects in a multi-view data set (i.e. multiple sources of data from the same set of subjects) where each view marginally follows a mixture model. In the case of two views, the dependence between them is captured by a cluster membership matrix parameter and we aim to learn the structure of this matrix (e.g. the zero pattern). First, we develop a penalized likelihood approach to estimate the sparsity pattern of the cluster membership matrix. For the specific case of block diagonal structures, we develop a constrained likelihood formulation where this matrix is constrained to be block diagonal up to permutations of the rows and columns. To enforce block diagonal constraints we propose a novel optimization approach based on the symmetric graph Laplacian. We demonstrate the performance of these methods through both simulations and applications to data sets from cancer genetics and neuroscience. Both methods naturally extend to multiple views.
- [29] arXiv:2012.15339 [pdf, other]
-
Title: Fast covariance parameter estimation of spatial Gaussian process models using neural networksComments: 12 pages, 5 figuresSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Applications (stat.AP); Computation (stat.CO)
Gaussian processes (GPs) are a popular model for spatially referenced data and allow descriptive statements, predictions at new locations, and simulation of new fields. Often a few parameters are sufficient to parameterize the covariance function, and maximum likelihood (ML) methods can be used to estimate these parameters from data. ML methods, however, are computationally demanding. For example, in the case of local likelihood estimation, even fitting covariance models on modest size windows can overwhelm typical computational resources for data analysis. This limitation motivates the idea of using neural network (NN) methods to approximate ML estimates. We train NNs to take moderate size spatial fields or variograms as input and return the range and noise-to-signal covariance parameters. Once trained, the NNs provide estimates with a similar accuracy compared to ML estimation and at a speedup by a factor of 100 or more. Although we focus on a specific covariance estimation problem motivated by a climate science application, this work can be easily extended to other, more complex, spatial problems and provides a proof-of-concept for this use of machine learning in computational statistics.
- [30] arXiv:2012.15452 [pdf, other]
-
Title: Perfect Gibbs Sampling of Order Constrained Non-IID Ordered Random Variates with Application to Bayesian Principal Components AnalysisComments: 18 pages, 20 figuresSubjects: Computation (stat.CO)
Order statistics arising from $m$ independent but not identically distributed random variables are typically constructed by arranging some $X_{1}, X_{2}, \ldots, X_{m}$, with $X_{i}$ having distribution function $F_{i}(x)$, in increasing order denoted as $X_{(1)} \leq X_{(2)} \leq \ldots \leq X_{(m)}$. In this case, $X_{(i)}$ is not necessarily associated with $F_{i}(x)$. Assuming one can simulate values from each distribution, one can generate such ``non-iid" order statistics by simulating $X_{i}$ from $F_{i}$, for $i=1,2,\ldots, m$, and simply putting them in order. In this paper, we consider the problem of simulating ordered values $X_{(1)}, X_{(2)}, \ldots, X_{(m)}$ such that the marginal distribution of $X_{(i)}$ is $F_{i}(x)$. This problem arises in Bayesian principal components analysis (BPCA) where the $X_{i}$ are ordered eigenvalues that are a posteriori independent but not identically distributed. In this paper, we propose a novel {\emph{coupling-from-the-past}} algorithm to ``perfectly" (up to computable order of accuracy) simulate such "order-constrained non-iid" order statistics. We demonstrate the effectiveness of our approach for several examples, including the BPCA problem.
- [31] arXiv:2012.15467 [pdf, other]
-
Title: Fast Global Convergence for Low-rank Matrix Recovery via Riemannian Gradient Descent with Random InitializationSubjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Optimization and Control (math.OC); Statistics Theory (math.ST)
In this paper, we propose a new global analysis framework for a class of low-rank matrix recovery problems on the Riemannian manifold. We analyze the global behavior for the Riemannian optimization with random initialization. We use the Riemannian gradient descent algorithm to minimize a least squares loss function, and study the asymptotic behavior as well as the exact convergence rate. We reveal a previously unknown geometric property of the low-rank matrix manifold, which is the existence of spurious critical points for the simple least squares function on the manifold. We show that under some assumptions, the Riemannian gradient descent starting from a random initialization with high probability avoids these spurious critical points and only converges to the ground truth in nearly linear convergence rate, i.e. $\mathcal{O}(\text{log}(\frac{1}{\epsilon})+ \text{log}(n))$ iterations to reach an $\epsilon$-accurate solution. We use two applications as examples for our global analysis. The first one is a rank-1 matrix recovery problem. The second one is the Gaussian phase retrieval problem. The second example only satisfies the weak isometry property, but has behavior similar to that of the first one except for an extra saddle set. Our convergence guarantee is nearly optimal and almost dimension-free, which fully explains the numerical observations. The global analysis can be potentially extended to other data problems with random measurement structures and empirical least squares loss functions.
- [32] arXiv:2012.15477 [pdf, other]
-
Title: Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate AnalysisComments: 32 pagesSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
We propose the particle dual averaging (PDA) method, which generalizes the dual averaging method in convex optimization to the optimization over probability distributions with quantitative runtime guarantee. The algorithm consists of an inner loop and outer loop: the inner loop utilizes the Langevin algorithm to approximately solve for a stationary distribution, which is then optimized in the outer loop. The method can thus be interpreted as an extension of the Langevin algorithm to naturally handle nonlinear functional on the probability space. An important application of the proposed method is the optimization of two-layer neural network in the mean field regime, which is theoretically attractive due to the presence of nonlinear feature learning, but quantitative convergence rate can be challenging to establish. We show that neural networks in the mean field limit can be globally optimized by PDA. Furthermore, we characterize the convergence rate by leveraging convex optimization theory in finite-dimensional spaces. Our theoretical results are supported by numerical simulations on neural networks with reasonable size.
- [33] arXiv:2012.15526 [pdf, ps, other]
-
Title: Asymptotics of sums of regression residuals under multiple ordering of regressorsSubjects: Statistics Theory (math.ST)
We prove theorems about the Gaussian asymptotics of an empirical bridge built from linear model regressors with multiple regressor ordering. We study the testing of the hypothesis of a linear model for the components of a random vector: one of the components is a linear combination of the others up to an error that does not depend on the other components of the random vector. The results of observations of independent copies of a random vector are sequentially ordered in ascending order of several of its components. The result is a sequence of vectors of higher dimension, consisting of induced order statistics (concomitants) corresponding to different orderings. For this sequence of vectors, without the assumption of a linear model for the components, we prove a lemma of weak convergence of the distributions of an appropriately centered and normalized process to a centered Gaussian process with almost surely continuous trajectories. Assuming a linear relationship of the components, standard least squares estimates are used to compute regression residuals, the difference between response values and the predicted ones by the linear model. We prove a theorem of weak convergence of the process of regression residuals under the necessary normalization to a centered Gaussian process. Then we prove a theorem of the same convergence for the empirical bridge, a self-centered and self-normalized process of regression residuals.
- [34] arXiv:2012.15550 [pdf, ps, other]
-
Title: Nonreversible MCMC from conditional invertible transforms: a complete recipe with convergence guaranteesAuthors: Achille Thin, Nikita Kotolevskii, Christophe Andrieu, Alain Durmus, Eric Moulines, Maxim PanovComments: Submitted to AISTAT 2021Subjects: Computation (stat.CO); Machine Learning (stat.ML)
Markov Chain Monte Carlo (MCMC) is a class of algorithms to sample complex and high-dimensional probability distributions. The Metropolis-Hastings (MH) algorithm, the workhorse of MCMC, provides a simple recipe to construct reversible Markov kernels. Reversibility is a tractable property which implies a less tractable but essential property here, invariance. Reversibility is however not necessarily desirable when considering performance. This has prompted recent interest in designing kernels breaking this property. At the same time, an active stream of research has focused on the design of novel versions of the MH kernel, some nonreversible, relying on the use of complex invertible deterministic transforms. While standard implementations of the MH kernel are well understood, aforementioned developments have not received the same systematic treatment to ensure their validity. This paper fills the gap by developing general tools to ensure that a class of nonreversible Markov kernels, possibly relying on complex transforms, has the desired invariance property and lead to convergent algorithms. This leads to a set of simple and practically verifiable conditions.
- [35] arXiv:2012.15611 [pdf, other]
-
Title: Semi-Parametric Estimation of Incubation and Generation Times by Means of Laguerre PolynomialsSubjects: Statistics Theory (math.ST)
In epidemics many interesting quantities, like the reproduction number, depend on the incubation period (time from infection to symptom onset) and/or the generation time (time until a new person is infected from another infected person). Therefore, estimation of the distribution of these two quantities is of distinct interest. However, this is a challenging problem since it is normally not possible to obtain precise observations of these two variables. Instead, in the beginning of a pandemic, it is possible to observe for infection pairs the time of symptom onset for both people as well as a window for infection of the first person (e.g. because of travel to a risk area). In this paper we suggest a simple semi-parametric sieve-estimation method based on Laguerre-Polynomials for estimation of these distributions. We provide detailed theory for consistency and illustrate the finite sample performance for small datasets via a simulation study.
- [36] arXiv:2012.15664 [pdf, other]
-
Title: Inference post Selection of Group-sparse Regression ModelsComments: 60 pages, 6 FiguresSubjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)
Conditional inference provides a rigorous approach to counter bias when data from automated model selections is reused for inference. We develop in this paper a statistically consistent Bayesian framework to assess uncertainties within linear models that are informed by grouped sparsities in covariates. Finding wide applications when genes, proteins, genetic variants, neuroimaging measurements are grouped respectively by their biological pathways, molecular functions, regulatory regions, cognitive roles, these models are selected through a useful class of group-sparse learning algorithms. An adjustment factor to account precisely for the selection of promising groups, deployed with a generalized version of Laplace-type approximations is the centerpiece of our new methods. Accommodating well known group-sparse models such as those selected by the Group LASSO, the overlapping Group LASSO, the sparse Group LASSO etc., we illustrate the efficacy of our methodology in extensive experiments and on data from a human neuroimaging application.
- [37] arXiv:2012.15678 [pdf, ps, other]
-
Title: On Gaussian Approximation for M-EstimatorComments: 49 pagesSubjects: Statistics Theory (math.ST)
This study develops a non-asymptotic Gaussian approximation theory for distributions of M-estimators, which are defined as maximizers of empirical criterion functions. In existing mathematical statistics literature, numerous studies have focused on approximating the distributions of the M-estimators for statistical inference. In contrast to the existing approaches, which mainly focus on limiting behaviors, this study employs a non-asymptotic approach, establishes abstract Gaussian approximation results for maximizers of empirical criteria, and proposes a Gaussian multiplier bootstrap approximation method. Our developments can be considered as an extension of the seminal works (Chernozhukov, Chetverikov and Kato (2013, 2014, 2015)) on the approximation theory for distributions of suprema of empirical processes toward their maximizers. Through this work, we shed new lights on the statistical theory of M-estimators. Our theory covers not only regular estimators, such as the least absolute deviations, but also some non-regular cases where it is difficult to derive or to approximate numerically the limiting distributions such as non-Donsker classes and cube root estimators.
- [38] arXiv:2012.15726 [pdf, other]
-
Title: Refined bounds for randomized experimental designSubjects: Statistics Theory (math.ST); Machine Learning (cs.LG); Methodology (stat.ME)
Experimental design is an approach for selecting samples among a given set so as to obtain the best estimator for a given criterion. In the context of linear regression, several optimal designs have been derived, each associated with a different criterion: mean square error, robustness, \emph{etc}. Computing such designs is generally an NP-hard problem and one can instead rely on a convex relaxation that considers probability distributions over the samples. Although greedy strategies and rounding procedures have received a lot of attention, straightforward sampling from the optimal distribution has hardly been investigated. In this paper, we propose theoretical guarantees for randomized strategies on E and G-optimal design. To this end, we develop a new concentration inequality for the eigenvalues of random matrices using a refined version of the intrinsic dimension that enables us to quantify the performance of such randomized strategies. Finally, we evidence the validity of our analysis through experiments, with particular attention on the G-optimal design applied to the best arm identification problem for linear bandits.
Cross-lists for Fri, 1 Jan 21
- [39] arXiv:2012.14453 (cross-list from cs.LG) [pdf, other]
-
Title: Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System HeterogeneitySubjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (stat.ML)
Federated Learning is a novel paradigm that involves learning from data samples distributed across a large network of clients while the data remains local. It is, however, known that federated learning is prone to multiple system challenges including system heterogeneity where clients have different computation and communication capabilities. Such heterogeneity in clients' computation speeds has a negative effect on the scalability of federated learning algorithms and causes significant slow-down in their runtime due to the existence of stragglers. In this paper, we propose a novel straggler-resilient federated learning method that incorporates statistical characteristics of the clients' data to adaptively select the clients in order to speed up the learning procedure. The key idea of our algorithm is to start the training procedure with faster nodes and gradually involve the slower nodes in the model training once the statistical accuracy of the data corresponding to the current participating nodes is reached. The proposed approach reduces the overall runtime required to achieve the statistical accuracy of data of all nodes, as the solution for each stage is close to the solution of the subsequent stage with more samples and can be used as a warm-start. Our theoretical results characterize the speedup gain in comparison to standard federated benchmarks for strongly convex objectives, and our numerical experiments also demonstrate significant speedups in wall-clock time of our straggler-resilient method compared to federated learning benchmarks.
- [40] arXiv:2012.14465 (cross-list from cs.LG) [pdf, other]
-
Title: Classification of Pathological and Normal Gait: A SurveySubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Applications (stat.AP)
Gait recognition is a term commonly referred to as an identification problem within the Computer Science field. There are a variety of methods and models capable of identifying an individual based on their pattern of ambulatory locomotion. By surveying the current literature on gait recognition, this paper seeks to identify appropriate metrics, devices, and algorithms for collecting and analyzing data regarding patterns and modes of ambulatory movement across individuals. Furthermore, this survey seeks to motivate interest in a broader scope of longitudinal analysis regarding the perturbations in gait across states (i.e. physiological, emotive, and/or cognitive states). More broadly, inferences to normal versus pathological gait patterns can be attributed, based on both longitudinal and non-longitudinal forms of classification. This may indicate promising research directions and experimental designs, such as creating algorithmic metrics for the quantification of fatigue, or models for forecasting episodic disorders. Furthermore, in conjunction with other measurements of physiological and environmental conditions, pathological gait classification might be applicable to inference for syndromic surveillance of infectious disease states or cognitive impairment.
- [41] arXiv:2012.14503 (cross-list from econ.GN) [pdf, other]
-
Title: Growth, development, and structural change at the firm-level: The example of the PR ChinaSubjects: General Economics (econ.GN); Other Statistics (stat.OT)
Understanding the microeconomic details of technological catch-up processes offers great potential for informing both innovation economics and development policy. We study the economic transition of the PR China from an agrarian country to a high-tech economy as one example for such a case. It is clear from past literature that rapidly rising productivity levels played a crucial role. However, the distribution of labor productivity in Chinese firms has not been comprehensively investigated and it remains an open question if this can be used to guide economic development. We analyze labor productivity and the dynamic change of labor productivity in firm-level data for the years 1998-2013 from the Chinese Industrial Enterprise Database. We demonstrate that both variables are conveniently modeled as L\'evy alpha-stable distributions, provide parameter estimates and analyze dynamic changes to this distribution. We find that the productivity gains were not due to super-star firms, but due to a systematic shift of the entire distribution with otherwise mostly unchanged characteristics. We also found an emerging right-skew in the distribution of labor productivity change. While there are significant differences between the 31 provinces and autonomous regions of the P.R. China, we also show that there are systematic relations between micro-level and province-level variables. We conclude with some implications of these findings for development policy.
- [42] arXiv:2012.14521 (cross-list from cs.IT) [pdf, other]
-
Title: Minorization via Mixed Volumes and Cover's Problem for General ChannelsAuthors: Jingbo LiuSubjects: Information Theory (cs.IT); Probability (math.PR); Statistics Theory (math.ST)
We propose a method for establishing lower bounds on the supremum of processes in terms of packing numbers by means of mixed-volume inequalities (the Alexandrov-Fenchel inequality). A simple and general bound in terms of packing numbers under the convex distance is derived, from which some known bounds on the Gaussian processes and the Rademacher processes can be recovered when the convex set is taken to be the ball or the hypercube. However, the main thrust for our study of this approach is to handle non-i.i.d.\ (noncanonical) processes (correspondingly, the convex set is not a product set). As an application, we give a complete solution to an open problem of Thomas Cover in 1987 about the capacity of a relay channel in the general discrete memoryless setting.
- [43] arXiv:2012.14540 (cross-list from cs.LG) [pdf, ps, other]
-
Title: Source Identification for Mixtures of Product DistributionsSubjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Signal Processing (eess.SP); Machine Learning (stat.ML)
We give an algorithm for source identification of a mixture of $k$ product distributions on $n$ bits. This is a fundamental problem in machine learning with many applications. Our algorithm identifies the source parameters of an identifiable mixture, given, as input, approximate values of multilinear moments (derived, for instance, from a sufficiently large sample), using $2^{O(k^2)} n^{O(k)}$ arithmetic operations. Our result is the first explicit bound on the computational complexity of source identification of such mixtures. The running time improves previous results by Feldman, O'Donnell, and Servedio (FOCS 2005) and Chen and Moitra (STOC 2019) that guaranteed only learning the mixture (without parametric identification of the source). Our analysis gives a quantitative version of a qualitative characterization of identifiable sources that is due to Tahmasebi, Motahari, and Maddah-Ali (ISIT 2018).
- [44] arXiv:2012.14559 (cross-list from q-bio.NC) [pdf, other]
-
Title: Correlation Across Environments Encoded by Hippocampal Place CellsSubjects: Neurons and Cognition (q-bio.NC); Applications (stat.AP)
The hippocampus is often attributed to episodic memory formation and storage in the mammalian brain; in particular, Alme et al. showed that hippocampal area CA3 forms statistically independent representations across a large number of environments, even if the environments share highly similar features. This lack of overlap between spatial maps indicates the large capacity of the CA3 circuitry. In this paper, we support the argument for the large capacity of the CA3 network. To do so, we replicate the key findings of Alme et al. and extend the results by perturbing the neural activity encodings with noise and conducting representation similarity analysis (RSA). We find that the correlations between firing rates are partially resistant to noise, and that the spatial representations across cells show similar patterns, even across different environments. Finally, we discuss some theoretical and practical implications of our results.
- [45] arXiv:2012.14595 (cross-list from cs.LG) [pdf, other]
-
Title: Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature SelectionSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
In the field of data mining, how to deal with high-dimensional data is an inevitable problem. Unsupervised feature selection has attracted more and more attention because it does not rely on labels. The performance of spectral-based unsupervised methods depends on the quality of constructed similarity matrix, which is used to depict the intrinsic structure of data. However, real-world data contain a large number of noise samples and features, making the similarity matrix constructed by original data cannot be completely reliable. Worse still, the size of similarity matrix expands rapidly as the number of samples increases, making the computational cost increase significantly. Inspired by principal component analysis, we propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_{2,p}$-norm regularization. The projection matrix, which is used for feature selection, is learned by minimizing the reconstruction error under the sparse constraint. Then, we present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically. Finally, extensive experiments on real-world data sets demonstrate the effectiveness of our proposed method.
- [46] arXiv:2012.14670 (cross-list from cs.LG) [pdf, ps, other]
-
Title: Fast Incremental Expectation Maximization for finite-sum optimization: nonasymptotic convergenceSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Fast Incremental Expectation Maximization (FIEM) is a version of the EM framework for large datasets. In this paper, we first recast FIEM and other incremental EM type algorithms in the {\em Stochastic Approximation within EM} framework. Then, we provide nonasymptotic bounds for the convergence in expectation as a function of the number of examples $n$ and of the maximal number of iterations $\kmax$. We propose two strategies for achieving an $\epsilon$-approximate stationary point, respectively with $\kmax = O(n^{2/3}/\epsilon)$ and $\kmax = O(\sqrt{n}/\epsilon^{3/2})$, both strategies relying on a random termination rule before $\kmax$ and on a constant step size in the Stochastic Approximation step. Our bounds provide some improvements on the literature. First, they allow $\kmax$ to scale as $\sqrt{n}$ which is better than $n^{2/3}$ which was the best rate obtained so far; it is at the cost of a larger dependence upon the tolerance $\epsilon$, thus making this control relevant for small to medium accuracy with respect to the number of examples $n$. Second, for the $n^{2/3}$-rate, the numerical illustrations show that thanks to an optimized choice of the step size and of the bounds in terms of quantities characterizing the optimization problem at hand, our results desig a less conservative choice of the step size and provide a better control of the convergence in expectation.
- [47] arXiv:2012.14674 (cross-list from cs.DM) [pdf, ps, other]
-
Title: Logical indetermination coupling: a method to minimize drawing matches and its applicationsComments: arXiv admin note: text overlap with arXiv:2007.08820Subjects: Discrete Mathematics (cs.DM); Information Theory (cs.IT); Social and Information Networks (cs.SI); Probability (math.PR); Statistics Theory (math.ST)
While justifying that independence is a canonic coupling, the authors show the existence of a second equilibrium to reduce the information conveyed from the margins to the joined distribution: the so-called indetermination. They use this null information property to apply indetermination to graph clustering. Furthermore, they break down a drawing under indetermination to emphasis it is the best construction to reduce couple matchings, meaning, the expected number of equal couples drawn in a row. Using this property, they notice that indetermination appears in two problems (Guessing and Task Partitioning) where couple matchings reduction is a key objective.
- [48] arXiv:2012.14755 (cross-list from cs.LG) [pdf, other]
-
Title: Improved Sample Complexity for Incremental Autonomous Exploration in MDPsComments: NeurIPS 2020Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
We investigate the exploration of an unknown environment when no reward function is provided. Building on the incremental exploration setting introduced by Lim and Auer [1], we define the objective of learning the set of $\epsilon$-optimal goal-conditioned policies attaining all states that are incrementally reachable within $L$ steps (in expectation) from a reference state $s_0$. In this paper, we introduce a novel model-based approach that interleaves discovering new states from $s_0$ and improving the accuracy of a model estimate that is used to compute goal-conditioned policies to reach newly discovered states. The resulting algorithm, DisCo, achieves a sample complexity scaling as $\tilde{O}(L^5 S_{L+\epsilon} \Gamma_{L+\epsilon} A \epsilon^{-2})$, where $A$ is the number of actions, $S_{L+\epsilon}$ is the number of states that are incrementally reachable from $s_0$ in $L+\epsilon$ steps, and $\Gamma_{L+\epsilon}$ is the branching factor of the dynamics over such states. This improves over the algorithm proposed in [1] in both $\epsilon$ and $L$ at the cost of an extra $\Gamma_{L+\epsilon}$ factor, which is small in most environments of interest. Furthermore, DisCo is the first algorithm that can return an $\epsilon/c_{\min}$-optimal policy for any cost-sensitive shortest-path problem defined on the $L$-reachable states with minimum cost $c_{\min}$. Finally, we report preliminary empirical results confirming our theoretical findings.
- [49] arXiv:2012.14820 (cross-list from econ.EM) [pdf, other]
-
Title: Bayesian analysis of seasonally cointegrated VAR modelAuthors: Justyna WróblewskaSubjects: Econometrics (econ.EM); Methodology (stat.ME)
The paper aims at developing the Bayesian seasonally cointegrated model for quarterly data. We propose the prior structure, derive the set of full conditional posterior distributions, and propose the sampling scheme. The identification of cointegrating spaces is obtained \emph{via} orthonormality restrictions imposed on vectors spanning them. In the case of annual frequency, the cointegrating vectors are complex, which should be taken into account when identifying them. The point estimation of the cointegrating spaces is also discussed. The presented methods are illustrated by a simulation experiment and are employed in the analysis of money and prices in the Polish economy.
- [50] arXiv:2012.14823 (cross-list from econ.EM) [pdf, ps, other]
-
Title: Bias-Aware Inference in Regularized Regression ModelsComments: 38 pagesSubjects: Econometrics (econ.EM); Methodology (stat.ME)
We consider inference on a regression coefficient under a constraint on the magnitude of the control coefficients. We show that a class of estimators based on an auxiliary regularized regression of the regressor of interest on control variables exactly solves a tradeoff between worst-case bias and variance. We derive "bias-aware" confidence intervals (CIs) based on these estimators, which take into account possible bias when forming the critical value. We show that these estimators and CIs are near-optimal in finite samples for mean squared error and CI length. Our finite-sample results are based on an idealized setting with normal regression errors with known homoskedastic variance, and we provide conditions for asymptotic validity with unknown and possibly heteroskedastic error distribution. Focusing on the case where the constraint on the magnitude of control coefficients is based on an $\ell_p$ norm ($p\ge 1$), we derive rates of convergence for optimal estimators and CIs under high-dimensional asymptotics that allow the number of regressors to increase more quickly than the number of observations.
- [51] arXiv:2012.14868 (cross-list from cs.LG) [pdf, ps, other]
-
Title: Minimum Excess Risk in Bayesian LearningSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)
We analyze the best achievable performance of Bayesian learning under generative models by defining and upper-bounding the minimum excess risk (MER): the gap between the minimum expected loss attainable by learning from data and the minimum expected loss that could be achieved if the model realization were known. The definition of MER provides a principled way to define different notions of uncertainties in Bayesian learning, including the aleatoric uncertainty and the minimum epistemic uncertainty. Two methods for deriving upper bounds for the MER are presented. The first method, generally suitable for Bayesian learning with a parametric generative model, upper-bounds the MER by the conditional mutual information between the model parameters and the quantity being predicted given the observed data. It allows us to quantify the rate at which the MER decays to zero as more data becomes available. The second method, particularly suitable for Bayesian learning with a parametric predictive model, relates the MER to the deviation of the posterior predictive distribution from the true predictive model, and further to the minimum estimation error of the model parameters from data. It explicitly shows how the uncertainty in model parameter estimation translates to the MER and to the final prediction uncertainty. We also extend the definition and analysis of MER to the setting with multiple parametric model families and the setting with nonparametric models. Along the discussions we draw some comparisons between the MER in Bayesian learning and the excess risk in frequentist learning.
- [52] arXiv:2012.14873 (cross-list from cs.LG) [pdf, other]
-
Title: Twin Neural Network RegressionSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are normally costly to produce, TNN regression intrinsically creates an ensemble of predictions of twice the size of the training set while only training a single neural network. Since ensembles have been shown to be more accurate than single models this property naturally transfers to TNN regression. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods. Furthermore, TNN regression is constrained by self-consistency conditions. We find that the violation of these conditions provides an estimate for the prediction uncertainty.
- [53] arXiv:2012.14878 (cross-list from cs.LG) [pdf, other]
-
Title: Growing Deep Forests Efficiently with Soft Routing and Learned ConnectivityComments: ICDM workshop 2018Journal-ref: ICDM Workshops 2018: 399-402Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Despite the latest prevailing success of deep neural networks (DNNs), several concerns have been raised against their usage, including the lack of intepretability the gap between DNNs and other well-established machine learning models, and the growingly expensive computational costs. A number of recent works [1], [2], [3] explored the alternative to sequentially stacking decision tree/random forest building blocks in a purely feed-forward way, with no need of back propagation. Since decision trees enjoy inherent reasoning transparency, such deep forest models can also facilitate the understanding of the internaldecision making process. This paper further extends the deep forest idea in several important aspects. Firstly, we employ a probabilistic tree whose nodes make probabilistic routing decisions, a.k.a., soft routing, rather than hard binary decisions.Besides enhancing the flexibility, it also enables non-greedy optimization for each tree. Second, we propose an innovative topology learning strategy: every node in the ree now maintains a new learnable hyperparameter indicating the probability that it will be a leaf node. In that way, the tree will jointly optimize both its parameters and the tree topology during training. Experiments on the MNIST dataset demonstrate that our empowered deep forests can achieve better or comparable performance than [1],[3] , with dramatically reduced model complexity. For example,our model with only 1 layer of 15 trees can perform comparably with the model in [3] with 2 layers of 2000 trees each.
- [54] arXiv:2012.14905 (cross-list from cs.LG) [pdf, other]
-
Title: Meta Learning Backpropagation And Improving ItSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Many concepts have been proposed for meta learning with neural networks (NNs), e.g., NNs that learn to control fast weights, hyper networks, learned learning rules, and meta recurrent neural networks (Meta RNNs). Our Variable Shared Meta Learning (VS-ML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms. A simple implementation of VS-ML called Variable Shared Meta RNN allows for implementing the backpropagation learning algorithm solely by running an RNN in forward-mode. It can even meta-learn new learning algorithms that improve upon backpropagation, generalizing to different datasets without explicit gradient calculation.
- [55] arXiv:2012.14941 (cross-list from econ.GN) [pdf]
-
Title: The wealth of nations and the health of populations: A quasi-experimental design of the impact of sovereign debt crises on child mortalityAuthors: Adel DaoudSubjects: General Economics (econ.GN); Other Statistics (stat.OT)
The wealth of nations and the health of populations are intimately strongly associated, yet the extent to which economic prosperity (GDP per capita) causes improved health remains disputed. The purpose of this article is to analyze the impact of sovereign debt crises (SDC) on child mortality, using a sample of 57 low- and middle-income countries surveyed by the Demographic and Health Survey between the years 1990 and 2015. These surveys supply 229 household data and containing about 3 million childbirth history records. This focus on SDC instead of GDP provides a quasi-experimental moment in which the influence of unobserved confounding is less than a moment analyzing the normal fluctuations of GDP. This study measures child mortality at six thresholds: neonatal, under-one (infant), under-two, under-three, under-four, and under-five mortality. Using a machine-learning (ML) model for causal inference, this study finds that while an SDC causes an adverse yet statistically insignificant effect on neonatal mortality, all other child mortality group samples are adversely affected between a probability of 0.12 to 0.14 (all statistically significant at the 95-percent threshold). Through this ML, this study also finds that the most important treatment heterogeneity moderator, in the entire adjustment set, is whether a child is born in a low-income country.
- [56] arXiv:2012.14954 (cross-list from cs.CR) [pdf]
-
Title: Privacy-Preserving Methods for Vertically Partitioned Incomplete DataJournal-ref: 2020 AMIA Annual Symposium ProceedingsSubjects: Cryptography and Security (cs.CR); Methodology (stat.ME)
Distributed health data networks that use information from multiple sources have drawn substantial interest in recent years. However, missing data are prevalent in such networks and present significant analytical challenges. The current state-of-the-art methods for handling missing data require pooling data into a central repository before analysis, which may not be possible in a distributed health data network. In this paper, we propose a privacy-preserving distributed analysis framework for handling missing data when data are vertically partitioned. In this framework, each institution with a particular data source utilizes the local private data to calculate necessary intermediate aggregated statistics, which are then shared to build a global model for handling missing data. To evaluate our proposed methods, we conduct simulation studies that clearly demonstrate that the proposed privacy-preserving methods perform as well as the methods using the pooled data and outperform several na\"ive methods. We further illustrate the proposed methods through the analysis of a real dataset. The proposed framework for handling vertically partitioned incomplete data is substantially more privacy-preserving than methods that require pooling of the data, since no individual-level data are shared, which can lower hurdles for collaboration across multiple institutions and build stronger public trust.
- [57] arXiv:2012.14961 (cross-list from cs.LG) [pdf, other]
-
Title: Towards Fair Deep Anomaly DetectionComments: Accepted for publication at the ACM Conference on Fairness, Accountability, and Transparency 2021 (ACM FAccT'21)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Anomaly detection aims to find instances that are considered unusual and is a fundamental problem of data science. Recently, deep anomaly detection methods were shown to achieve superior results particularly in complex data such as images. Our work focuses on deep one-class classification for anomaly detection which learns a mapping only from the normal samples. However, the non-linear transformation performed by deep learning can potentially find patterns associated with social bias. The challenge with adding fairness to deep anomaly detection is to ensure both making fair and correct anomaly predictions simultaneously. In this paper, we propose a new architecture for the fair anomaly detection approach (Deep Fair SVDD) and train it using an adversarial network to de-correlate the relationships between the sensitive attributes and the learned representations. This differs from how fairness is typically added namely as a regularizer or a constraint. Further, we propose two effective fairness measures and empirically demonstrate that existing deep anomaly detection methods are unfair. We show that our proposed approach can remove the unfairness largely with minimal loss on the anomaly detection performance. Lastly, we conduct an in-depth analysis to show the strength and limitations of our proposed model, including parameter analysis, feature visualization, and run-time analysis.
- [58] arXiv:2012.14966 (cross-list from cs.LG) [pdf, other]
-
Title: Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear MapsAuthors: Tri Dao, Nimit S. Sohoni, Albert Gu, Matthew Eichhorn, Amit Blonder, Megan Leszczynski, Atri Rudra, Christopher RéComments: International Conference on Learning Representations (ICLR) 2020 spotlightSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Modern neural network architectures use structured linear transformations, such as low-rank matrices, sparse matrices, permutations, and the Fourier transform, to improve inference speed and reduce memory usage compared to general linear maps. However, choosing which of the myriad structured transformations to use (and its associated parameterization) is a laborious task that requires trading off speed, space, and accuracy. We consider a different approach: we introduce a family of matrices called kaleidoscope matrices (K-matrices) that provably capture any structured matrix with near-optimal space (parameter) and time (arithmetic operation) complexity. We empirically validate that K-matrices can be automatically learned within end-to-end pipelines to replace hand-crafted procedures, in order to improve model quality. For example, replacing channel shuffles in ShuffleNet improves classification accuracy on ImageNet by up to 5%. K-matrices can also simplify hand-engineered pipelines -- we replace filter bank feature computation in speech data preprocessing with a learnable kaleidoscope layer, resulting in only 0.4% loss in accuracy on the TIMIT speech recognition task. In addition, K-matrices can capture latent structure in models: for a challenging permuted image classification task, a K-matrix based representation of permutations is able to learn the right latent structure and improves accuracy of a downstream convolutional model by over 9%. We provide a practically efficient implementation of our approach, and use K-matrices in a Transformer network to attain 36% faster end-to-end inference speed on a language translation task.
- [59] arXiv:2012.15000 (cross-list from cs.LG) [pdf, other]
-
Title: DeepSphere: a graph-based spherical CNNComments: published at ICLR'20, this https URLSubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
Designing a convolution for a spherical neural network requires a delicate tradeoff between efficiency and rotation equivariance. DeepSphere, a method based on a graph representation of the sampled sphere, strikes a controllable balance between these two desiderata. This contribution is twofold. First, we study both theoretically and empirically how equivariance is affected by the underlying graph with respect to the number of vertices and neighbors. Second, we evaluate DeepSphere on relevant problems. Experiments show state-of-the-art performance and demonstrates the efficiency and flexibility of this formulation. Perhaps surprisingly, comparison with previous work suggests that anisotropic filters might be an unnecessary price to pay. Our code is available at https://github.com/deepsphere
- [60] arXiv:2012.15005 (cross-list from cs.LG) [pdf, other]
-
Title: Infer-AVAE: An Attribute Inference Model Based on Adversarial Variational AutoencoderSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Facing the sparsity of user attributes on social networks, attribute inference aims at inferring missing attributes based on existing data and additional information such as social connections between users. Recently, Variational Autoencoders (VAEs) have been successfully applied to solve the problem in a semi-supervised way. However, the latent representations learned by the encoder contain either insufficient or useless information: i) MLPs can successfully reconstruct the input data but fail in completing missing part, ii) GNNs merge information according to social connections but suffer from over-smoothing, which is a common problem with GNNs. Moreover, existing methods neglect regulating the decoder, as a result, it lacks adequate inference ability and faces severe overfitting. To address the above issues, we propose an attribute inference model based on adversarial VAE (Infer-AVAE). Our model deliberately unifies MLPs and GNNs in encoder to learn dual latent representations: one contains only the observed attributes of each user, the other converges extra information from the neighborhood. Then, an adversarial network is trained to leverage the differences between the two representations and adversarial training is conducted to guide GNNs using MLPs for robust representations. What's more, mutual information constraint is introduced in loss function to specifically train the decoder as a discriminator. Thus, it can make better use of auxiliary information in the representations for attribute inference. Based on real-world social network datasets, experimental results demonstrate that our model averagely outperforms state-of-art by 7.0% in accuracy.
- [61] arXiv:2012.15035 (cross-list from cs.HC) [pdf, other]
-
Title: Measuring an adaptive change in human decision-making from AI: Application to evaluate changes after AlphaGoSubjects: Human-Computer Interaction (cs.HC); General Economics (econ.GN); Applications (stat.AP)
Across a growing number of domains, human experts are expected to learn from and adapt to AI with superior decision-making abilities. But how can we quantify such human adaptation to AI? Using a Reinforcement Learning framework in the game of Go, we develop a simple measure of human adaptation to AI and test its usefulness in two case studies. In Case Study 1, we analyze 1.3 million move decisions made by human experts and find that a positive form of adaptation to AI (learning) occurred after the experts could observe AI's reasoning processes rather than mere actions of AI. In Case Study 2, we use our measure to detect a negative form of adaptation to AI, cheating by getting help from AI in a professional match between human experts. We discuss our measure's applications in domains other than Go, especially in domains in which AI's decision-making ability will likely surpass that of human experts.
- [62] arXiv:2012.15036 (cross-list from cs.LG) [pdf, other]
-
Title: SGD Distributional Dynamics of Three Layer Neural NetworksSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
With the rise of big data analytics, multi-layer neural networks have surfaced as one of the most powerful machine learning methods. However, their theoretical mathematical properties are still not fully understood. Training a neural network requires optimizing a non-convex objective function, typically done using stochastic gradient descent (SGD). In this paper, we seek to extend the mean field results of Mei et al. (2018) from two-layer neural networks with one hidden layer to three-layer neural networks with two hidden layers. We will show that the SGD dynamics is captured by a set of non-linear partial differential equations, and prove that the distributions of weights in the two hidden layers are independent. We will also detail exploratory work done based on simulation and real-world data.
- [63] arXiv:2012.15046 (cross-list from math.OC) [pdf, other]
-
Title: Risk Guarantees for End-to-End Prediction and Optimization ProcessesSubjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
Prediction models are often employed in estimating parameters of optimization models. Despite the fact that in an end-to-end view, the real goal is to achieve good optimization performance, the prediction performance is measured on its own. While it is usually believed that good prediction performance in estimating the parameters will result in good subsequent optimization performance, formal theoretical guarantees on this are notably lacking. In this paper, we explore conditions that allow us to explicitly describe how the prediction performance governs the optimization performance. Our weaker condition allows for an asymptotic convergence result, while our stronger condition allows for exact quantification of the optimization performance in terms of the prediction performance. In general, verification of these conditions is a non-trivial task. Nevertheless, we show that our weaker condition is equivalent to the well-known Fisher consistency concept from the learning theory literature. This then allows us to easily check our weaker condition for several loss functions. We also establish that the squared error loss function satisfies our stronger condition. Consequently, we derive the exact theoretical relationship between prediction performance measured with the squared loss, as well as a class of symmetric loss functions, and the subsequent optimization performance. In a computational study on portfolio optimization, fractional knapsack and multiclass classification problems, we compare the optimization performance of using of several prediction loss functions (some that are Fisher consistent and some that are not) and demonstrate that lack of consistency of the loss function can indeed have a detrimental effect on performance.
- [64] arXiv:2012.15059 (cross-list from cs.LG) [pdf, other]
-
Title: Ensembles of Localised Models for Time Series ForecastingComments: 29 pages, 5 figures, 5 tablesSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
With large quantities of data typically available nowadays, forecasting models that are trained across sets of time series, known as Global Forecasting Models (GFM), are regularly outperforming traditional univariate forecasting models that work on isolated series. As GFMs usually share the same set of parameters across all time series, they often have the problem of not being localised enough to a particular series, especially in situations where datasets are heterogeneous. We study how ensembling techniques can be used with generic GFMs and univariate models to solve this issue. Our work systematises and compares relevant current approaches, namely clustering series and training separate submodels per cluster, the so-called ensemble of specialists approach, and building heterogeneous ensembles of global and local models. We fill some gaps in the approaches and generalise them to different underlying GFM model types. We then propose a new methodology of clustered ensembles where we train multiple GFMs on different clusters of series, obtained by changing the number of clusters and cluster seeds. Using Feed-forward Neural Networks, Recurrent Neural Networks, and Pooled Regression models as the underlying GFMs, in our evaluation on six publicly available datasets, the proposed models are able to achieve significantly higher accuracy than baseline GFM models and univariate forecasting methods.
- [65] arXiv:2012.15085 (cross-list from cs.LG) [pdf, other]
-
Title: Is Pessimism Provably Efficient for Offline RL?Comments: 53 pages, 3 figuresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Statistics Theory (math.ST); Machine Learning (stat.ML)
We study offline reinforcement learning (RL), which aims to learn an optimal policy based on a dataset collected a priori. Due to the lack of further interactions with the environment, offline RL suffers from the insufficient coverage of the dataset, which eludes most existing theoretical analysis. In this paper, we propose a pessimistic variant of the value iteration algorithm (PEVI), which incorporates an uncertainty quantifier as the penalty function. Such a penalty function simply flips the sign of the bonus function for promoting exploration in online RL, which makes it easily implementable and compatible with general function approximators.
Without assuming the sufficient coverage of the dataset, we establish a data-dependent upper bound on the suboptimality of PEVI for general Markov decision processes (MDPs). When specialized to linear MDPs, it matches the information-theoretic lower bound up to multiplicative factors of the dimension and horizon. In other words, pessimism is not only provably efficient but also minimax optimal. In particular, given the dataset, the learned policy serves as the ``best effort'' among all policies, as no other policies can do better. Our theoretical analysis identifies the critical role of pessimism in eliminating a notion of spurious correlation, which emerges from the ``irrelevant'' trajectories that are less covered by the dataset and not informative for the optimal policy. - [66] arXiv:2012.15103 (cross-list from cs.LG) [pdf, other]
-
Title: Explanations of Machine Learning predictions: a mandatory step for its application to Operational ProcessesSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
In the global economy, credit companies play a central role in economic development, through their activity as money lenders. This important task comes with some drawbacks, mainly the risk of the debtors not being able to repay the provided credit. Therefore, Credit Risk Modelling (CRM), namely the evaluation of the probability that a debtor will not repay the due amount, plays a paramount role. Statistical approaches have been successfully exploited since long, becoming the most used methods for CRM. Recently, also machine and deep learning techniques have been applied to the CRM task, showing an important increase in prediction quality and performances. However, such techniques usually do not provide reliable explanations for the scores they come up with. As a consequence, many machine and deep learning techniques fail to comply with western countries' regulations such as, for example, GDPR. In this paper we suggest to use LIME (Local Interpretable Model-agnostic Explanations) technique to tackle the explainability problem in this field, we show its employment on a real credit-risk dataset and eventually discuss its soundness and the necessary improvements to guarantee its adoption and compliance with the task.
- [67] arXiv:2012.15112 (cross-list from cs.CY) [pdf, other]
-
Title: Web Routineness and Limits of Predictability: Investigating Demographic and Behavioral Differences Using Web Tracking DataComments: 12 pages, 8 figures. To be published in the proceedings of the International AAAI Conference on Web and Social Media (ICWSM) 2021Subjects: Computers and Society (cs.CY); Information Theory (cs.IT); Physics and Society (physics.soc-ph); Applications (stat.AP)
Understanding human activities and movements on the Web is not only important for computational social scientists but can also offer valuable guidance for the design of online systems for recommendations, caching, advertising, and personalization. In this work, we demonstrate that people tend to follow routines on the Web, and these repetitive patterns of web visits increase their browsing behavior's achievable predictability. We present an information-theoretic framework for measuring the uncertainty and theoretical limits of predictability of human mobility on the Web. We systematically assess the impact of different design decisions on the measurement. We apply the framework to a web tracking dataset of German internet users. Our empirical results highlight that individual's routines on the Web make their browsing behavior predictable to 85% on average, though the value varies across individuals. We observe that these differences in the users' predictabilities can be explained to some extent by their demographic and behavioral attributes.
- [68] arXiv:2012.15115 (cross-list from cs.CL) [pdf, other]
-
Title: Joint Verification and Reranking for Open Fact Checking Over TablesAuthors: Michael Schlichtkrull, Vladimir Karpukhin, Barlas Oğuz, Mike Lewis, Wen-tau Yih, Sebastian RiedelSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Machine Learning (stat.ML)
Structured information is an important knowledge source for automatic verification of factual claims. Nevertheless, the majority of existing research into this task has focused on textual data, and the few recent inquiries into structured data have been for the closed-domain setting where appropriate evidence for each claim is assumed to have already been retrieved. In this paper, we investigate verification over structured data in the open-domain setting, introducing a joint reranking-and-verification model which fuses evidence documents in the verification component. Our open-domain model achieves performance comparable to the closed-domain state-of-the-art on the TabFact dataset, and demonstrates performance gains from the inclusion of multiple tables as well as a significant improvement over a heuristic retrieval baseline.
- [69] arXiv:2012.15203 (cross-list from eess.SP) [pdf, other]
-
Title: Learning to Optimize Energy Efficiency in Energy Harvesting Wireless Sensor NetworksComments: 5 pages, 4 figures. Under review at IEEE Wireless Communications LettersSubjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
We study wireless power transmission by an energy source to multiple energy harvesting nodes with the aim to maximize the energy efficiency. The source transmits energy to the nodes using one of the available power levels in each time slot and the nodes transmit information back to the energy source using the harvested energy. The source does not have any channel state information and it only knows whether a received codeword from a given node was successfully decoded or not. With this limited information, the source has to learn the optimal power level that maximizes the energy efficiency of the network. We model the problem as a stochastic Multi-Armed Bandits problem and develop an Upper Confidence Bound based algorithm, which learns the optimal transmit power of the energy source that maximizes the energy efficiency. Numerical results validate the performance guarantees of the proposed algorithm and show significant gains compared to the benchmark schemes.
- [70] arXiv:2012.15231 (cross-list from cs.LG) [pdf, other]
-
Title: A Novel Resampling Technique for Imbalanced Dataset OptimizationSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Despite the enormous amount of data, particular events of interest can still be quite rare. Classification of rare events is a common problem in many domains, such as fraudulent transactions, malware traffic analysis and network intrusion detection. Many studies have been developed for malware detection using machine learning approaches on various datasets, but as far as we know only the MTA-KDD'19 dataset has the peculiarity of updating the representative set of malicious traffic on a daily basis. This daily updating is the added value of the dataset, but it translates into a potential due to the class imbalance problem that the RRw-Optimized MTA-KDD'19 will occur. We capture difficulties of class distribution in real datasets by considering four types of minority class examples: safe, borderline, rare and outliers. In this work, we developed two versions of Generative Silhouette Resampling 1-Nearest Neighbour (G1Nos) oversampling algorithms for dealing with class imbalance problem. The first module of G1Nos algorithms performs a coefficient-based instance selection silhouette identifying the critical threshold of Imbalance Degree. (ID), the second module generates synthetic samples using a SMOTE-like oversampling algorithm. The balancing of the classes is done by our G1Nos algorithms to re-establish the proportions between the two classes of the used dataset. The experimental results show that our oversampling algorithm work better than the other two SOTA methodologies in all the metrics considered.
- [71] arXiv:2012.15259 (cross-list from cs.LG) [pdf, other]
-
Title: A Maximal Correlation Approach to Imposing Fairness in Machine LearningAuthors: Joshua Lee, Yuheng Bu, Prasanna Sattigeri, Rameswar Panda, Gregory Wornell, Leonid Karlinsky, Rogerio FerisComments: 9 Pages 4 FiguresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (stat.ML)
As machine learning algorithms grow in popularity and diversify to many industries, ethical and legal concerns regarding their fairness have become increasingly relevant. We explore the problem of algorithmic fairness, taking an information-theoretic view. The maximal correlation framework is introduced for expressing fairness constraints and shown to be capable of being used to derive regularizers that enforce independence and separation-based fairness criteria, which admit optimization algorithms for both discrete and continuous variables which are more computationally efficient than existing algorithms. We show that these algorithms provide smooth performance-fairness tradeoff curves and perform competitively with state-of-the-art methods on both discrete datasets (COMPAS, Adult) and continuous datasets (Communities and Crimes).
- [72] arXiv:2012.15332 (cross-list from cs.CL) [pdf, other]
-
Title: kōan: A Corrected CBOW ImplementationSubjects: Computation and Language (cs.CL); Machine Learning (stat.ML)
It is a common belief in the NLP community that continuous bag-of-words (CBOW) word embeddings tend to underperform skip-gram (SG) embeddings. We find that this belief is founded less on theoretical differences in their training objectives but more on faulty CBOW implementations in standard software libraries such as the official implementation word2vec.c and Gensim. We show that our correct implementation of CBOW yields word embeddings that are fully competitive with SG on various intrinsic and extrinsic tasks while being more than three times as fast to train. We release our implementation, k\=oan, at https://github.com/bloomberg/koan.
- [73] arXiv:2012.15367 (cross-list from econ.EM) [pdf, other]
-
Title: Assessing the Sensitivity of Synthetic Control Treatment Effect Estimates to Misspecification ErrorComments: 32 pages, 6 figuresSubjects: Econometrics (econ.EM); Methodology (stat.ME)
We propose a sensitivity analysis for Synthetic Control (SC) treatment effect estimates to interrogate the assumption that the SC method is well-specified, namely that choosing weights to minimize pre-treatment prediction error yields accurate predictions of counterfactual post-treatment outcomes. Our data-driven procedure recovers the set of treatment effects consistent with the assumption that the misspecification error incurred by the SC method is at most the observable misspecification error incurred when using the SC estimator to predict the outcomes of some control unit. We show that under one definition of misspecification error, our procedure provides a simple, geometric motivation for comparing the estimated treatment effect to the distribution of placebo residuals to assess estimate credibility. When applied to several canonical studies that use the SC method, our procedure demonstrates that the signs of most of those results are relatively robust.
- [74] arXiv:2012.15368 (cross-list from physics.soc-ph) [pdf, other]
-
Title: More crime in cities? On the scaling laws of crime and the inadequacy of per capita rankings -- a cross-country studyAuthors: Marcos OliveiraComments: 19 pages, 5 figuresSubjects: Physics and Society (physics.soc-ph); Computers and Society (cs.CY); Applications (stat.AP)
Objectives: To evaluate the relationship between population size and number of crimes in cities across twelve countries and assess the impact of per capita measurements on crime analyses, depending on offense type.
Methods: We use data on burglaries and thefts at the city level and evaluate the relationship between crime numbers and population size using probabilistic scaling analysis. We estimate the growth exponent of each offense type and use Kendall rank correlation to assess the impact of a linear growth assumption (i.e., per-capita analysis) on cities rankings.
Result: In nine out of eleven countries, theft increases superlinearly with population size; in two of them, it increases linearly. In eight out of ten countries, burglary increases linearly with population size; in two of them, it increases superlinearly. In nonlinear scenarios, using per capita rates to rank cities produces substantially different rankings from rankings adjusted for population size.
Conclusions: Comparing cities using per capita crime rates (e.g., crime per 100,000 people per year) assumes that crime increases linearly with population size. Our findings indicate, however, that this assumption is unfounded, implying that one should be cautious when using per capita rankings. When crime increases nonlinearly with population, per capita rates do not remove population effects. The contrasting crime growth of burglary and theft also suggests that different crime dynamics at the local level lead to different macro-level features in cities. - [75] arXiv:2012.15411 (cross-list from math.OC) [pdf, other]
-
Title: Constrained and Composite Optimization via Adaptive Sampling MethodsComments: 26 pages, 13 figuresSubjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems in which the objective function is stochastic and the constraints are deterministic. The method proposed in this paper
is a proximal gradient method that can also be applied to the composite optimization problem min f(x) + h(x), where f is stochastic and h is convex (but not necessarily differentiable). Adaptive sampling methods employ a mechanism for gradually improving the quality of the gradient approximation so as to keep computational cost to a minimum. The mechanism commonly employed in unconstrained optimization is no longer reliable in the constrained or composite optimization settings because it is based on pointwise decisions that cannot correctly predict the quality of the proximal gradient step. The method proposed in this paper measures the result of a complete step to determine if the gradient approximation is accurate enough; otherwise a more accurate gradient is generated and a new step is computed. Convergence results are established both for strongly convex and general convex f. Numerical experiments are presented to illustrate the practical behavior of the method. - [76] arXiv:2012.15458 (cross-list from math.OC) [pdf, other]
-
Title: Differentiable Programming à la MoreauSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
The notion of a Moreau envelope is central to the analysis of first-order optimization algorithms for machine learning. Yet, it has not been developed and extended to be applied to a deep network and, more broadly, to a machine learning system with a differentiable programming implementation. We define a compositional calculus adapted to Moreau envelopes and show how to integrate it within differentiable programming. The proposed framework casts in a mathematical optimization framework several variants of gradient back-propagation related to the idea of the propagation of virtual targets.
- [77] arXiv:2012.15480 (cross-list from cs.LG) [pdf, other]
-
Title: Likelihood Ratio Exponential FamiliesComments: NeurIPS Workshop on Deep Learning through Information GeometrySubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)
The exponential family is well known in machine learning and statistical physics as the maximum entropy distribution subject to a set of observed constraints, while the geometric mixture path is common in MCMC methods such as annealed importance sampling. Linking these two ideas, recent work has interpreted the geometric mixture path as an exponential family of distributions to analyse the thermodynamic variational objective (TVO).
We extend likelihood ratio exponential families to include solutions to rate-distortion (RD) optimization, the information bottleneck (IB) method, and recent rate-distortion-classification approaches which combine RD and IB. This provides a common mathematical framework for understanding these methods via the conjugate duality of exponential families and hypothesis testing. Further, we collect existing results to provide a variational representation of intermediate RD or TVO distributions as a minimizing an expectation of KL divergences. This solution also corresponds to a size-power tradeoff using the likelihood ratio test and the Neyman Pearson lemma. In thermodynamic integration bounds such as the TVO, we identify the intermediate distribution whose expected sufficient statistics match the log partition function. - [78] arXiv:2012.15483 (cross-list from cs.LG) [pdf, other]
-
Title: Why do classifier accuracies show linear trends under distribution shift?Comments: 14 pages, 11 figuresSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Several recent studies observed that when classification models are evaluated on two different data distributions, the models' accuracies on one distribution are approximately a linear function of their accuracies on another distribution. We offer an explanation for these observations based on two assumptions that can be assessed empirically: (1) certain events have similar probabilities under the two distributions; (2) the probability that a lower accuracy model correctly classifies a data point sampled from one distribution when a higher accuracy model classifies it incorrectly is small.
- [79] arXiv:2012.15566 (cross-list from cs.LG) [pdf, other]
-
Title: Robust Asymmetric Learning in POMDPsSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Policies for partially observed Markov decision processes can be efficiently learned by imitating policies for the corresponding fully observed Markov decision processes. Unfortunately, existing approaches for this kind of imitation learning have a serious flaw: the expert does not know what the trainee cannot see, and so may encourage actions that are sub-optimal, even unsafe, under partial information. We derive an objective to instead train the expert to maximize the expected reward of the imitating agent policy, and use it to construct an efficient algorithm, adaptive asymmetric DAgger (A2D), that jointly trains the expert and the agent. We show that A2D produces an expert policy that the agent can safely imitate, in turn outperforming policies learned by imitating a fixed expert.
- [80] arXiv:2012.15584 (cross-list from cs.LG) [pdf, other]
-
Title: Combinatorial Pure Exploration with Full-bandit Feedback and Beyond: Solving Combinatorial Optimization under Uncertainty with Limited ObservationComments: Preprint of an Invited Review ArticleSubjects: Machine Learning (cs.LG); Discrete Mathematics (cs.DM); Data Structures and Algorithms (cs.DS); Social and Information Networks (cs.SI); Machine Learning (stat.ML)
Combinatorial optimization is one of the fundamental research fields that has been extensively studied in theoretical computer science and operations research. When developing an algorithm for combinatorial optimization, it is commonly assumed that parameters such as edge weights are exactly known as inputs. However, this assumption may not be fulfilled since input parameters are often uncertain or initially unknown in many applications such as recommender systems, crowdsourcing, communication networks, and online advertisement. To resolve such uncertainty, the problem of combinatorial pure exploration of multi-armed bandits (CPE) and its variants have recieved increasing attention. Earlier work on CPE has studied the semi-bandit feedback or assumed that the outcome from each individual edge is always accessible at all rounds. However, due to practical constraints such as a budget ceiling or privacy concern, such strong feedback is not always available in recent applications. In this article, we review recently proposed techniques for combinatorial pure exploration problems with limited feedback.
- [81] arXiv:2012.15716 (cross-list from econ.EM) [pdf, other]
-
Title: Assessing Sensitivity to Unconfoundedness: Estimation and InferenceComments: 37 pages with 41 page appendixSubjects: Econometrics (econ.EM); Methodology (stat.ME)
This paper provides a set of methods for quantifying the robustness of treatment effects estimated using the unconfoundedness assumption (also known as selection on observables or conditional independence). Specifically, we estimate and do inference on bounds on various treatment effect parameters, like the average treatment effect (ATE) and the average effect of treatment on the treated (ATT), under nonparametric relaxations of the unconfoundedness assumption indexed by a scalar sensitivity parameter c. These relaxations allow for limited selection on unobservables, depending on the value of c. For large enough c, these bounds equal the no assumptions bounds. Using a non-standard bootstrap method, we show how to construct confidence bands for these bound functions which are uniform over all values of c. We illustrate these methods with an empirical application to effects of the National Supported Work Demonstration program. We implement these methods in a companion Stata module for easy use in practice.
- [82] arXiv:2012.15802 (cross-list from cs.LG) [pdf, ps, other]
-
Title: The Sample Complexity of Robust Covariance TestingSubjects: Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
We study the problem of testing the covariance matrix of a high-dimensional Gaussian in a robust setting, where the input distribution has been corrupted in Huber's contamination model. Specifically, we are given i.i.d. samples from a distribution of the form $Z = (1-\epsilon) X + \epsilon B$, where $X$ is a zero-mean and unknown covariance Gaussian $\mathcal{N}(0, \Sigma)$, $B$ is a fixed but unknown noise distribution, and $\epsilon>0$ is an arbitrarily small constant representing the proportion of contamination. We want to distinguish between the cases that $\Sigma$ is the identity matrix versus $\gamma$-far from the identity in Frobenius norm.
In the absence of contamination, prior work gave a simple tester for this hypothesis testing task that uses $O(d)$ samples. Moreover, this sample upper bound was shown to be best possible, within constant factors. Our main result is that the sample complexity of covariance testing dramatically increases in the contaminated setting. In particular, we prove a sample complexity lower bound of $\Omega(d^2)$ for $\epsilon$ an arbitrarily small constant and $\gamma = 1/2$. This lower bound is best possible, as $O(d^2)$ samples suffice to even robustly {\em learn} the covariance. The conceptual implication of our result is that, for the natural setting we consider, robust hypothesis testing is at least as hard as robust estimation. - [83] arXiv:2012.15814 (cross-list from cs.LG) [pdf, other]
-
Title: Language-Mediated, Object-Centric Representation LearningComments: First two authors contributed equally; last two authors contributed equally. Project page: this https URLSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
We present Language-mediated, Object-centric Representation Learning (LORL), a paradigm for learning disentangled, object-centric scene representations from vision and language. LORL builds upon recent advances in unsupervised object segmentation, notably MONet and Slot Attention. While these algorithms learn an object-centric representation just by reconstructing the input image, LORL enables them to further learn to associate the learned representations to concepts, i.e., words for object categories, properties, and spatial relationships, from language input. These object-centric concepts derived from language facilitate the learning of object-centric representations. LORL can be integrated with various unsupervised segmentation algorithms that are language-agnostic. Experiments show that the integration of LORL consistently improves the object segmentation performance of MONet and Slot Attention on two datasets via the help of language. We also show that concepts learned by LORL, in conjunction with segmentation algorithms such as MONet, aid downstream tasks such as referring expression comprehension.
- [84] arXiv:2012.15816 (cross-list from cs.LG) [pdf, other]
-
Title: Fairness in Machine LearningJournal-ref: Recent Trends in Learning From Data. Studies in Computational Intelligence, vol 896. Springer, Cham, 2020Subjects: Machine Learning (cs.LG); Computers and Society (cs.CY); Machine Learning (stat.ML)
Machine learning based systems are reaching society at large and in many aspects of everyday life. This phenomenon has been accompanied by concerns about the ethical issues that may arise from the adoption of these technologies. ML fairness is a recently established area of machine learning that studies how to ensure that biases in the data and model inaccuracies do not lead to models that treat individuals unfavorably on the basis of characteristics such as e.g. race, gender, disabilities, and sexual or political orientation. In this manuscript, we discuss some of the limitations present in the current reasoning about fairness and in methods that deal with it, and describe some work done by the authors to address them. More specifically, we show how causal Bayesian networks can play an important role to reason about and deal with fairness, especially in complex unfairness scenarios. We describe how optimal transport theory can be used to develop methods that impose constraints on the full shapes of distributions corresponding to different sensitive attributes, overcoming the limitation of most approaches that approximate fairness desiderata by imposing constraints on the lower order moments or other functions of those distributions. We present a unified framework that encompasses methods that can deal with different settings and fairness criteria, and that enjoys strong theoretical guarantees. We introduce an approach to learn fair representations that can generalize to unseen tasks. Finally, we describe a technique that accounts for legal restrictions about the use of sensitive attributes.
- [85] arXiv:2012.15829 (cross-list from cs.LG) [pdf, other]
-
Title: Continuity of Generalized Entropy and Statistical LearningAuthors: Aolin XuSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST)
We study the continuity property of the generalized entropy as a functional of the underlying probability distribution, defined with an action space and a loss function, and use this property to answer the basic questions in statistical learning theory, the excess risk analyses for various learning methods. We first derive upper and lower bounds for the entropy difference of two distributions in terms of several commonly used $f$-divergences, the Wasserstein distance, and a distance that depends on the action space and the loss function. Examples are given along with the discussion of each general result, comparisons are made with the existing entropy difference bounds, and new mutual information upper bounds are derived based on the new results. We then apply the entropy difference bounds to the theory of statistical learning. It is shown that the excess risks in the two popular learning paradigms, the frequentist learning and the Bayesian learning, both can be studied with the continuity property of different forms of the generalized entropy. The analysis is then extended to the continuity of generalized conditional entropy. The extension provides performance bounds for Bayes decision making with mismatched distributions. It also leads to excess risk bounds for a third paradigm of learning, where the decision rule is optimally designed under the projection of the empirical distribution to a predefined family of distributions. We thus establish a unified method of excess risk analysis for the three major paradigms of statistical learning, through the continuity of generalized entropy.
- [86] arXiv:2012.15863 (cross-list from cs.SI) [pdf, other]
-
Title: Empirically Classifying Network MechanismsComments: 5 pages, 2 figuresSubjects: Social and Information Networks (cs.SI); Machine Learning (cs.LG); Computation (stat.CO)
Network models are used to study interconnected systems across many physical, biological, and social disciplines. Such models often assume a particular network-generating mechanism, which when fit to data produces estimates of mechanism-specific parameters that describe how systems function. For instance, a social network model might assume new individuals connect to others with probability proportional to their number of pre-existing connections ('preferential attachment'), and then estimate the disparity in interactions between famous and obscure individuals with similar qualifications. However, without a means of testing the relevance of the assumed mechanism, conclusions from such models could be misleading. Here we introduce a simple empirical approach which can mechanistically classify arbitrary network data. Our approach compares empirical networks to model networks from a user-provided candidate set of mechanisms, and classifies each network--with high accuracy--as originating from either one of the mechanisms or none of them. We tested 373 empirical networks against five of the most widely studied network mechanisms and found that most (228) were unlike any of these mechanisms. This raises the possibility that some empirical networks arise from mixtures of mechanisms. We show that mixtures are often unidentifiable because different mixtures can produce functionally equivalent networks. In such systems, which are governed by multiple mechanisms, our approach can still accurately predict out-of-sample functional properties.
Replacements for Fri, 1 Jan 21
- [87] arXiv:1502.04206 (replaced) [pdf, other]
-
Title: Combining probability distributions: Extending the logarithmic pooling approachComments: Massively updated manuscript; submitted for publicationSubjects: Methodology (stat.ME)
- [88] arXiv:1602.04941 (replaced) [pdf, other]
-
Title: On Perfect Classification and Clustering for Gaussian ProcessesComments: 52 pages, 2 figuresSubjects: Statistics Theory (math.ST)
- [89] arXiv:1703.04730 (replaced) [pdf, other]
-
Title: Understanding Black-box Predictions via Influence FunctionsComments: International Conference on Machine Learning, 2017. (This version adds more historical references and fixes typos.)Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
- [90] arXiv:1706.01865 (replaced) [pdf, other]
-
Title: Estimating Shape Parameters of Piecewise Linear-Quadratic ProblemsComments: 15 pages, 6 figuresSubjects: Machine Learning (stat.ML)
- [91] arXiv:1710.06611 (replaced) [pdf, other]
-
Title: On community structure validation in real networksSubjects: Applications (stat.AP); Social and Information Networks (cs.SI); Physics and Society (physics.soc-ph); Methodology (stat.ME)
- [92] arXiv:1801.00332 (replaced) [pdf, ps, other]
-
Title: Confidence set for group membershipComments: 41 pages, supplementary materials (37 pages)Subjects: Econometrics (econ.EM); Methodology (stat.ME)
- [93] arXiv:1803.03627 (replaced) [pdf, other]
-
Title: A local depth measure for general dataSubjects: Methodology (stat.ME)
- [94] arXiv:1806.07585 (replaced) [pdf, other]
- [95] arXiv:1808.03591 (replaced) [pdf, other]
-
Title: How Complex is your classification problem? A survey on measuring classification complexityComments: Survey paperSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [96] arXiv:1901.05191 (replaced) [pdf, other]
-
Title: Multivariate mixed membership modeling: Inferring domain-specific risk profilesSubjects: Applications (stat.AP)
- [97] arXiv:1903.03825 (replaced) [pdf, other]
-
Title: Interpolation Consistency Training for Semi-Supervised LearningComments: Extended version of IJCAI 2019 paper. Semi-supervised Learning, Deep Learning, Neural Networks. All the previous results are unchanged; we added new theoretical and empirical resultsSubjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
- [98] arXiv:1903.05895 (replaced) [pdf, other]
-
Title: Learning Fast Algorithms for Linear Transforms Using Butterfly FactorizationsComments: International Conference on Machine Learning (ICML) 2019Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [99] arXiv:1904.04480 (replaced) [pdf, other]
-
Title: On the Adaptivity of Stochastic Gradient-Based OptimizationComments: Accepted by SIAM Journal on Optimization; 54 pagesSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
- [100] arXiv:1905.09710 (replaced) [pdf, other]
-
Title: Inverse Reinforcement Learning in Contextual MDPsSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [101] arXiv:1905.13229 (replaced) [pdf, ps, other]
-
Title: Private Hypothesis SelectionComments: Appeared in NeurIPS 2019. Final version to appear in IEEE Transactions on Information TheorySubjects: Data Structures and Algorithms (cs.DS); Cryptography and Security (cs.CR); Machine Learning (cs.LG); Machine Learning (stat.ML)
- [102] arXiv:1906.02179 (replaced) [pdf, other]
-
Title: Bayesian Active Learning With Abstention FeedbacksComments: Poster presented at 2019 ICML Workshop on Human in the Loop Learning 2019 (non-archival). arXiv admin note: substantial text overlap with arXiv:1705.08481Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Machine Learning (stat.ML)
- [103] arXiv:1906.05890 (replaced) [pdf, other]
-
Title: Gradient Descent Maximizes the Margin of Homogeneous Neural NetworksComments: 52 pages, 9 figures; Published in ICLR 2020; Corrected typos and added a few referencesSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
- [104] arXiv:1906.06589 (replaced) [pdf, other]
-
Title: Membership Privacy for Machine Learning Models Through Knowledge TransferComments: To Appear in the 35th AAAI Conference on Artificial Intelligence, 2021Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [105] arXiv:1906.08509 (replaced) [pdf, other]
-
Title: Online A-Optimal Design and Active Linear RegressionComments: 29 pages, 5 figuresSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
- [106] arXiv:1907.04027 (replaced) [pdf, other]
-
Title: Iteratively Reweighted $\ell_1$-Penalized Robust RegressionComments: 62 pagesSubjects: Statistics Theory (math.ST); Machine Learning (stat.ML)
- [107] arXiv:1907.06133 (replaced) [pdf, other]
-
Title: An Assumption-Free Exact Test For Fixed-Design Linear Models With Exchangeable ErrorsComments: Accepted by Biometrika; 46 pagesSubjects: Methodology (stat.ME)
- [108] arXiv:1908.05745 (replaced) [pdf, other]
-
Title: A Bayesian marked spatial point processes model for basketball shot chartJournal-ref: Journal of Quantitative Analysis in Sports (2020)Subjects: Applications (stat.AP); Computation (stat.CO)
- [109] arXiv:1909.01783 (replaced) [pdf, ps, other]
-
Title: Oracle Efficient Private Non-Convex OptimizationSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)
- [110] arXiv:1909.05236 (replaced) [pdf, other]
-
Title: Safe Policy Improvement with an Estimated Baseline PolicyComments: Published at AAMAS 2020Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
- [111] arXiv:1910.08475 (replaced) [pdf, other]
-
Title: On Warm-Starting Neural Network TrainingJournal-ref: 2020 Advances in Neural Information Processing SystemsSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
- [112] arXiv:1910.08520 (replaced) [pdf, other]
-
Title: Optimization Hierarchy for Fair Statistical Decision ProblemsSubjects: Statistics Theory (math.ST); Optimization and Control (math.OC)
- [113] arXiv:1911.07123 (replaced) [pdf, other]
-
Title: Graph-Revised Convolutional NetworkComments: ECML-PKDD 2020Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [114] arXiv:1912.10774 (replaced) [pdf, other]
-
Title: Probability Assessments of an Ice-Free Arctic: Comparing Statistical and Climate Model ProjectionsSubjects: Applications (stat.AP); Econometrics (econ.EM)
- [115] arXiv:1912.13213 (replaced) [pdf, other]
-
Title: A Modern Introduction to Online LearningAuthors: Francesco OrabonaComments: Fixed more typos, added more history bits, added local norms bounds for OMD and FTRLSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
- [116] arXiv:2001.05965 (replaced) [pdf, ps, other]
-
Title: The Elliptical Ornstein-Uhlenbeck ProcessSubjects: Methodology (stat.ME); Earth and Planetary Astrophysics (astro-ph.EP); Signal Processing (eess.SP); Data Analysis, Statistics and Probability (physics.data-an); Applications (stat.AP)
- [117] arXiv:2001.10523 (replaced) [pdf, other]
-
Title: Multi-class Gaussian Process Classification with Noisy InputsAuthors: Carlos Villacampa-Calvo, Bryan Zaldivar, Eduardo C. Garrido-Merchán, Daniel Hernández-LobatoSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
- [118] arXiv:2001.11936 (replaced) [pdf, other]
-
Title: An Autonomous Intrusion Detection System Using an Ensemble of Advanced LearnersComments: 5 pagesSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [119] arXiv:2002.00041 (replaced) [pdf, other]
-
Title: On Implicit Regularization in $β$-VAEsComments: ICML 2020; Final version, including appendixSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [120] arXiv:2002.00253 (replaced) [pdf, ps, other]
-
Title: Bandits with Knapsacks beyond the Worst-Case AnalysisComments: The initial version, titled "Advances in Bandits with Knapsacks", was published on arxiv.org in Jan'20. The present version improves both upper and lower bounds, deriving Theorem 3.2(ii) and Theorem 4.2. Moreover, it simplifies the algorithm and analysis in the main result, and fixes several issues in the lower boundsSubjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Machine Learning (stat.ML)
- [121] arXiv:2002.09051 (replaced) [pdf, ps, other]
-
Title: An Elementary Approach to Convergence Guarantees of Optimization Algorithms for Deep NetworksComments: The changes from v1 to v2 include i) slightly more general results; ii) slightly more concise proofs; iii) highway and residual networks; iv) implicitly defined network layers; v) additional algorithm boxes and illustration figuresSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [122] arXiv:2003.06121 (replaced) [pdf, other]
-
Title: When are Non-Parametric Methods Robust?Comments: accepted to ICML 2020Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [123] arXiv:2003.11435 (replaced) [pdf, other]
-
Title: Preferential Batch Bayesian OptimizationAuthors: Eero Siivola, Akash Kumar Dhaka, Michael Riis Andersen, Javier Gonzalez, Pablo Garcia Moreno, Aki VehtariSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [124] arXiv:2003.11941 (replaced) [pdf, other]
-
Title: AliExpress Learning-To-Rank: Maximizing Online Model Performance without Going OnlineAuthors: Guangda Huzhang, Zhen-Jia Pang, Yongqing Gao, Yawen Liu, Weijie Shen, Wen-Ji Zhou, Qing Da, An-Xiang Zeng, Han Yu, Yang Yu, Zhi-Hua ZhouSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
- [125] arXiv:2003.12206 (replaced) [pdf, other]
-
Title: Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)Authors: Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivière, Alina Beygelzimer, Florence d'Alché-Buc, Emily Fox, Hugo LarochelleComments: To appear at JMLR, 16 pages + AppendixSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [126] arXiv:2004.05718 (replaced) [pdf, other]
-
Title: Principal Neighbourhood Aggregation for Graph NetsComments: 34th Conference on Neural Information Processing Systems (NeurIPS 2020)Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
- [127] arXiv:2004.07802 (replaced) [pdf, other]
-
Title: Geometry-Aware Gradient Algorithms for Neural Architecture SearchSubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE); Optimization and Control (math.OC); Machine Learning (stat.ML)
- [128] arXiv:2004.12012 (replaced) [pdf, other]
-
Title: Integrative Bayesian models using Post-selective Inference: a case study in RadiogenomicsComments: 44 pages, 7 FiguresSubjects: Applications (stat.AP); Computation (stat.CO); Methodology (stat.ME)
- [129] arXiv:2005.02196 (replaced) [pdf, other]
-
Title: Measuring the Discrepancy between Conditional Distributions: Methods, Properties and ApplicationsComments: manuscript accepted at IJCAI 20; added additional notes on computational complexity and auto-differentiable property; code is available at this https URLSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)
- [130] arXiv:2005.06255 (replaced) [pdf, other]
-
Title: Detecting an Odd Restless Markov Arm with a Trembling HandComments: 46 pages, 1 figure, 1 table. Revision to an earlier submitted versionSubjects: Information Theory (cs.IT); Machine Learning (stat.ML)
- [131] arXiv:2005.12979 (replaced) [pdf, other]
-
Title: Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-Start UsersComments: TOIS 2021Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG); Social and Information Networks (cs.SI); Machine Learning (stat.ML)
- [132] arXiv:2005.14462 (replaced) [pdf, other]
-
Title: Estimation of Semi-Markov Multi-state Models: A Comparison of the Sojourn Times and Transition Intensities ApproachesSubjects: Methodology (stat.ME)
- [133] arXiv:2006.02479 (replaced) [pdf, other]
-
Title: Least $k$th-Order and Rényi Generative Adversarial NetworksSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)
- [134] arXiv:2006.03258 (replaced) [pdf, other]
-
Title: Learned Factor Graphs for Inference from Stationary Time SequencesSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)
- [135] arXiv:2006.05158 (replaced) [pdf, ps, other]
-
Title: Homomorphic Sensing of Subspace ArrangementsComments: 16 pagesSubjects: Machine Learning (cs.LG); Algebraic Geometry (math.AG); Machine Learning (stat.ML)
- [136] arXiv:2006.06010 (replaced) [pdf, other]
-
Title: Higher-order interactions in statistical physics and machine learning: A model-independent solution to the inverse problem at equilibriumComments: 25 pages, 25 figures. Comments welcomeSubjects: Methodology (stat.ME); Statistical Mechanics (cond-mat.stat-mech); High Energy Physics - Lattice (hep-lat); Computational Physics (physics.comp-ph); Machine Learning (stat.ML)
- [137] arXiv:2006.07493 (replaced) [pdf, other]
-
Title: Bayesian Additive Regression Trees with Model TreesSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
- [138] arXiv:2006.10222 (replaced) [pdf, other]
-
Title: Class-Attentive Diffusion Network for Semi-Supervised ClassificationComments: Accepted to AAAI 2021Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [139] arXiv:2006.13365 (replaced) [pdf, other]
-
Title: Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified FrameworkAuthors: Mehdi Ali, Max Berrendorf, Charles Tapley Hoyt, Laurent Vermue, Mikhail Galkin, Sahand Sharifzadeh, Asja Fischer, Volker Tresp, Jens LehmannSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
- [140] arXiv:2006.15412 (replaced) [pdf, ps, other]
-
Title: Submodular Combinatorial Information Measures with Applications in Machine LearningComments: Will appear in the 32nd International Conference on Algorithmic Learning Theory, ALT 2021Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Optimization and Control (math.OC); Machine Learning (stat.ML)
- [141] arXiv:2007.01281 (replaced) [pdf, other]
-
Title: Efficient estimation of the ANOVA mean dimension, with an application to neural net classificationSubjects: Methodology (stat.ME); Machine Learning (cs.LG); Numerical Analysis (math.NA)
- [142] arXiv:2007.04612 (replaced) [pdf, other]
-
Title: Concept Bottleneck ModelsAuthors: Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy LiangComments: Edited for clarity from the ICML 2020 versionSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [143] arXiv:2007.09601 (replaced) [pdf, other]
-
Title: Hypersolvers: Toward Fast Continuous-Depth ModelsSubjects: Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)
- [144] arXiv:2007.11401 (replaced) [pdf, ps, other]
-
Title: Wasserstein Statistics in One-dimensional Location-Scale ModelComments: arXiv admin note: substantial text overlap with arXiv:2003.05479Subjects: Statistics Theory (math.ST)
- [145] arXiv:2008.03869 (replaced) [pdf]
-
Title: Individualized Prediction of COVID-19 Adverse outcomes with MLHOSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Quantitative Methods (q-bio.QM)
- [146] arXiv:2008.04195 (replaced) [pdf, other]
-
Title: An improved convergence analysis for decentralized online stochastic non-convex optimizationSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Systems and Control (eess.SY); Machine Learning (stat.ML)
- [147] arXiv:2008.07707 (replaced) [pdf, other]
-
Title: RTFN: Robust Temporal Feature NetworkComments: 10pages, 6 figuresSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [148] arXiv:2008.09887 (replaced) [pdf, other]
-
Title: Data Programming using Semi-Supervision and Subset SelectionAuthors: Ayush Maheshwari, Oishik Chatterjee, KrishnaTeja Killamsetty, Rishabh Iyer, Ganesh RamakrishnanComments: 11 pages, 2 figures, 13 tablesSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [149] arXiv:2008.10466 (replaced) [pdf, ps, other]
-
Title: Column $\ell_{2,0}$-norm regularized factorization model of low-rank matrix recovery and its computationSubjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
- [150] arXiv:2009.01411 (replaced) [pdf, other]
-
Title: Learning from Protein Structure with Geometric Vector PerceptronsSubjects: Biomolecules (q-bio.BM); Machine Learning (cs.LG); Machine Learning (stat.ML)
- [151] arXiv:2009.02183 (replaced) [pdf, other]
-
Title: On the implementation of a global optimization method for mixed-variable problemsAuthors: Giacomo NanniciniSubjects: Machine Learning (cs.LG); Discrete Mathematics (cs.DM); Optimization and Control (math.OC); Machine Learning (stat.ML)
- [152] arXiv:2009.02861 (replaced) [pdf, other]
-
Title: Constant Regret Re-solving Heuristics for Price-based Revenue ManagementComments: Revised version. Improved regret upper bounds to O(1) and extension/generalization to multiple products and heteroscedastic demand noisesSubjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
- [153] arXiv:2009.05150 (replaced) [pdf, other]
-
Title: Inference for high-dimensional exchangeable arraysSubjects: Econometrics (econ.EM); Statistics Theory (math.ST)
- [154] arXiv:2009.06962 (replaced) [pdf, other]
-
Title: Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal MixupComments: Published at ICML 2020Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
- [155] arXiv:2009.08405 (replaced) [pdf, ps, other]
-
Title: Bayesian Matrix Completion for Hypothesis TestingSubjects: Applications (stat.AP)
- [156] arXiv:2009.08435 (replaced) [pdf, other]
-
Title: Large Norms of CNN Layers Do Not Hurt Adversarial RobustnessComments: Accepted to AAAI-2021; 15 pages, 4 figures; v4: corrected typoSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
- [157] arXiv:2010.01184 (replaced) [pdf, other]
-
Title: Covariate Shift Adaptation in High-Dimensional and Divergent DistributionsSubjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Methodology (stat.ME)
- [158] arXiv:2010.04789 (replaced) [pdf]
-
Title: Bayesian Characterization of Uncertainties Surrounding Fluvial Flood Hazard EstimatesAuthors: Sanjib Sharma, Ganesh Raj Ghimire, Rocky Talchabhadel, Jeeban Panthi, Benjamin Seiyon Lee, Fengyun Sun, Rupesh Baniya, Tirtha Raj AdhikariSubjects: Applications (stat.AP)
- [159] arXiv:2010.06993 (replaced) [pdf, ps, other]
-
Title: Weight Squeezing: Reparameterization for Extreme Compression and Fast InferenceSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Machine Learning (stat.ML)
- [160] arXiv:2010.14449 (replaced) [pdf, other]
-
Title: On Principal Component Regression in a High-Dimensional Error-in-Variables SettingSubjects: Statistics Theory (math.ST); Machine Learning (cs.LG); Machine Learning (stat.ML)
- [161] arXiv:2011.03073 (replaced) [pdf, ps, other]
-
Title: Conditional quantile estimators: A small sample theorySubjects: Econometrics (econ.EM); Probability (math.PR); Statistics Theory (math.ST)
- [162] arXiv:2011.04622 (replaced) [pdf, ps, other]
-
Title: On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State SpacesComments: 76 pages. The short version of this work appears in NeurIPS 2020Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Statistics Theory (math.ST); Machine Learning (stat.ML)
- [163] arXiv:2011.14990 (replaced) [pdf, other]
-
Title: Multiscale Comparative ConnectomicsAuthors: Vivek Gopalakrishnan, Jaewon Chung, Eric Bridgeford, Benjamin D. Pedigo, Jesús Arroyo, Lucy Upchurch, G. Allan Johnson, Nian Wang, Youngser Park, Carey E. Priebe, Joshua T. VogelsteinComments: 29 pages, 12 figuresSubjects: Neurons and Cognition (q-bio.NC); Methodology (stat.ME)
- [164] arXiv:2012.00394 (replaced) [pdf, other]
-
Title: Semi-Mechanistic Bayesian Modeling of COVID-19 with Renewal ProcessesSubjects: Applications (stat.AP); Methodology (stat.ME)
- [165] arXiv:2012.02119 (replaced) [pdf, other]
-
Title: Robustly Learning Mixtures of $k$ Arbitrary GaussiansAuthors: Ainesh Bakshi, Ilias Diakonikolas, He Jia, Daniel M. Kane, Pravesh K. Kothari, Santosh S. VempalaComments: This version fixes an issue with a technical statement (Fact 2.35 in the previous arXiv version, replaced by Lemmas 2.38 and 2.41 in the current version) that claimed a bound on the variance of a more general class of distributions than what is needed in our approachSubjects: Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
- [166] arXiv:2012.04171 (replaced) [pdf, other]
-
Title: Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorizationAuthors: Joshua C. Chang, Patrick Fletcher, Jungmin Han, Ted L. Chang, Shashaank Vattikuti, Bart Desmet, Ayah Zirikly, Carson C. ChowComments: Fixed typo in Eq 2Subjects: Machine Learning (cs.LG); Quantitative Methods (q-bio.QM); Machine Learning (stat.ML)
- [167] arXiv:2012.05199 (replaced) [pdf, other]
-
Title: A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein DistanceSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
- [168] arXiv:2012.07348 (replaced) [pdf, other]
-
Title: Bandit Learning in Decentralized Matching MarketsComments: 23 pagesSubjects: Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT); Multiagent Systems (cs.MA); Machine Learning (stat.ML)
- [169] arXiv:2012.09922 (replaced) [pdf, ps, other]
-
Title: Individually Conditional Individual Mutual Information Bound on Generalization ErrorSubjects: Information Theory (cs.IT); Machine Learning (stat.ML)
- [170] arXiv:2012.11849 (replaced) [pdf, other]
-
Title: Selective Forgetting of Deep Networks at a Finer Level than SamplesComments: 8+2 pages, 6 figures, accepted to AAAI RSEMLSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
- [171] arXiv:2012.13962 (replaced) [pdf, other]
-
Title: A Tutorial on Sparse Gaussian Processes and Variational InferenceSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
- [172] arXiv:2012.14331 (replaced) [pdf, other]
-
Title: A method to integrate and classify normal distributionsComments: 9 pages, 5 figuresSubjects: Machine Learning (stat.ML); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
[ showing up to 2000 entries per page: fewer | more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, stat, recent, 2101, contact, help (Access key information)