Machine Learning

New submissions

• New submissions
• Cross-lists
• Replacements

New submissions for Wed, 6 Nov 19

[1]  arXiv:1911.01425 [pdf, other]

Title: Improved BiGAN training with marginal likelihood equalization

Authors: Pablo Sánchez-Martín, Pablo M. Olmos, Fernando Perez-Cruz

Subjects: Machine Learning (stat.ML); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

We propose a novel training procedure for improving the performance of generative adversarial networks (GANs), especially to bidirectional GANs. First, we enforce that the empirical distribution of the inverse inference network matches the prior distribution, which favors the generator network reproducibility on the seen samples. Second, we have found that the marginal log-likelihood of the samples shows a severe overrepresentation of a certain type of samples. To address this issue, we propose to train the bidirectional GAN using a non-uniform sampling for the mini-batch selection, resulting in improved quality and variety in generated samples measured quantitatively and by visual inspection. We illustrate our new procedure with the well-known CIFAR10, Fashion MNIST and CelebA datasets.

[2]  arXiv:1911.01429 [pdf, other]

Title: The frontier of simulation-based inference

Authors: Kyle Cranmer, Johann Brehmer, Gilles Louppe

Comments: 8 pages, 3 figures, proceedings for the Sackler Colloquia at the US National Academy of Sciences

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces giving new momentum to the field. Finally, we describe how the frontier is expanding so that a broad audience can appreciate the profound change these developments may have on science.

[3]  arXiv:1911.01469 [pdf, ps, other]

Title: Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry

Authors: Andre Wibisono

Subjects: Machine Learning (stat.ML); Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Machine Learning (cs.LG)

We study the Proximal Langevin Algorithm (PLA) for sampling from a probability distribution $\nu = e^{-f}$ on $\mathbb{R}^n$ under isoperimetry. We prove a convergence guarantee for PLA in Kullback-Leibler (KL) divergence when $\nu$ satisfies log-Sobolev inequality (LSI) and $f$ has bounded second and third derivatives. This improves on the result for the Unadjusted Langevin Algorithm (ULA), and matches the fastest known rate for sampling under LSI (without Metropolis filter) with a better dependence on the LSI constant. We also prove convergence guarantees for PLA in R\'enyi divergence of order $q > 1$ when the biased limit satisfies either LSI or Poincar\'e inequality.

[4]  arXiv:1911.01483 [pdf, other]

Title: Statistical Inference for Model Parameters in Stochastic Gradient Descent via Batch Means

Authors: Yi Zhu, Jing Dong

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)

Statistical inference of true model parameters based on stochastic gradient descent (SGD) has started receiving attention in recent years. In this paper, we study a simple algorithm to construct asymptotically valid confidence regions for model parameters using the batch means method. The main idea is to cancel out the covariance matrix which is hard/costly to estimate. In the process of developing the algorithm, we establish process-level function central limit theorem for Polyak-Ruppert averaging based SGD estimators. We also extend the batch means method to accommodate more general batch size specifications.

[5]  arXiv:1911.01535 [pdf, other]

Title: Scalable Deep Generative Relational Models with High-Order Node Dependence

Authors: Xuhui Fan, Bin Li, Scott Anthony Sisson, Caoyuan Li, Ling Chen

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

We propose a probabilistic framework for modelling and exploring the latent structure of relational data. Given feature information for the nodes in a network, the scalable deep generative relational model (SDREM) builds a deep network architecture that can approximate potential nonlinear mappings between nodes' feature information and the nodes' latent representations. Our contribution is two-fold: (1) We incorporate high-order neighbourhood structure information to generate the latent representations at each node, which vary smoothly over the network. (2) Due to the Dirichlet random variable structure of the latent representations, we introduce a novel data augmentation trick which permits efficient Gibbs sampling. The SDREM can be used for large sparse networks as its computational cost scales with the number of positive links. We demonstrate its competitive performance through improved link prediction performance on a range of real-world datasets.

[6]  arXiv:1911.01929 [pdf, other]

Title: GP-ALPS: Automatic Latent Process Selection for Multi-Output Gaussian Process Models

Authors: Pavel Berkovich, Eric Perim, Wessel Bruinsma

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)

A simple and widely adopted approach to extend Gaussian processes (GPs) to multiple outputs is to model each output as a linear combination of a collection of shared, unobserved latent GPs. An issue with this approach is choosing the number of latent processes and their kernels. These choices are typically done manually, which can be time consuming and prone to human biases. We propose Gaussian Process Automatic Latent Process Selection (GP-ALPS), which automatically chooses the latent processes by turning off those that do not meaningfully contribute to explaining the data. We develop a variational inference scheme, assess the quality of the variational posterior by comparing it against the gold standard MCMC, and demonstrate the suitability of GP-ALPS in a set of preliminary experiments.

Cross-lists for Wed, 6 Nov 19

[7]  arXiv:1911.01419 (cross-list from cs.RO) [pdf, other]

Title: On Solving the 2-Dimensional Greedy Shooter Problem for UAVs

Authors: Loren Anderson, Sahitya Senapathy

Comments: 7 pages, 13 figures

Subjects: Robotics (cs.RO); Machine Learning (cs.LG); Machine Learning (stat.ML)

Unmanned Aerial Vehicles (UAVs), autonomously-guided aircraft, are widely used for tasks involving surveillance and reconnaissance. A version of the pursuit-evasion problems centered around UAVs and its variants has been extensively studied in recent years due to numerous breakthroughs in AI. We present an approach to UAV pursuit-evasion in a 2D aerial-engagement environment using reinforcement learning (RL), a machine learning paradigm concerned with goal-oriented algorithms. In this work, a UAV wielding the greedy shooter strategy engages with a UAV trained using deep Q-learning techniques. Simulated results show that the latter UAV wins every engagement in which the UAVs are suffciently separated during initialization. This approach highlights an exhaustive and robust application of reinforcement learning to pursuit-evasion that provides insight into effective strategies for UAV flight and interaction.

[8]  arXiv:1911.01458 (cross-list from eess.IV) [pdf, other]

Title: Dual-domain Cascade of U-nets for Multi-channel Magnetic Resonance Image Reconstruction

Authors: Roberto Souza, Mariana Bento, Nikita Nogovitsyn, Kevin J. Chung, R. Marc Lebel, Richard Frayne

Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG); Medical Physics (physics.med-ph); Machine Learning (stat.ML)

The U-net is a deep-learning network model that has been used to solve a number of inverse problems. In this work, the concatenation of two-element U-nets, termed the W-net, operating in k-space (K) and image (I) domains, were evaluated for multi-channel magnetic resonance (MR) image reconstruction. The two element network combinations were evaluated for the four possible image-k-space domain configurations: a) W-net II, b) W-net KK, c) W-net IK, and d) W-net KI were evaluated. Selected promising four element networks (WW-nets) were also examined. Two configurations of each network were compared: 1) Each coil channel processed independently, and 2) all channels processed simultaneously. One hundred and eleven volumetric, T1-weighted, 12-channel coil k-space datasets were used in the experiments. Normalized root mean squared error, peak signal to noise ratio, visual information fidelity and visual inspection were used to assess the reconstructed images against the fully sampled reference images. Our results indicated that networks that operate solely in the image domain are better suited when processing individual channels of multi-channel data independently. Dual domain methods are more advantageous when simultaneously reconstructing all channels of multi-channel data. Also, the appropriate cascade of U-nets compared favorably (p < 0.01) to the previously published, state-of-the-art Deep Cascade model in in three out of four experiments.

[9]  arXiv:1911.01462 (cross-list from cs.LG) [pdf, ps, other]

Title: Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals

Authors: Surbhi Goel, Sushrut Karmalkar, Adam Klivans

Comments: To appear in NeurIPS 2019 (Spotlight)

Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Machine Learning (stat.ML)

We consider the problem of computing the best-fitting ReLU with respect to square-loss on a training set when the examples have been drawn according to a spherical Gaussian distribution (the labels can be arbitrary). Let $\mathsf{opt} < 1$ be the population loss of the best-fitting ReLU. We prove:
1. Finding a ReLU with square-loss $\mathsf{opt} + \epsilon$ is as hard as the problem of learning sparse parities with noise, widely thought to be computationally intractable. This is the first hardness result for learning a ReLU with respect to Gaussian marginals, and our results imply -{\emph unconditionally}- that gradient descent cannot converge to the global minimum in polynomial time.
2. There exists an efficient approximation algorithm for finding the best-fitting ReLU that achieves error $O(\mathsf{opt}^{2/3})$. The algorithm uses a novel reduction to noisy halfspace learning with respect to $0/1$ loss.
Prior work due to Soltanolkotabi [Sol17] showed that gradient descent can find the best-fitting ReLU with respect to Gaussian marginals, if the training set is exactly labeled by a ReLU.

[10]  arXiv:1911.01468 (cross-list from cs.LG) [pdf, other]

Title: Auditing and Achieving Intersectional Fairness in Classification Problems

Authors: Giulio Morina, Viktoriia Oliinyk, Julian Waton, Ines Marusic, Konstantinos Georgatzis

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (stat.ML)

Machine learning algorithms are extensively used to make increasingly more consequential decisions, so that achieving optimal predictive performance can no longer be the only focus. This paper explores intersectional fairness, that is fairness when intersections of multiple sensitive attributes -- such as race, age, nationality, etc. -- are considered. Previous research has mainly been focusing on fairness with respect to a single sensitive attribute, with intersectional fairness being comparatively less studied despite its critical importance for modern machine learning applications. We introduce intersectional fairness metrics by extending prior work, and provide different methodologies to audit discrimination in a given dataset or model outputs. Secondly, we develop novel post-processing techniques to mitigate any detected bias in a classification model. Our proposed methodology does not rely on any assumptions regarding the underlying model and aims at guaranteeing fairness while preserving good predictive performance. Finally, we give guidance on a practical implementation, showing how the proposed methods perform on a real-world dataset.

[11]  arXiv:1911.01484 (cross-list from cs.LG) [pdf, other]

Title: Improving Supervised Phase Identification Through the Theory of Information Losses

Authors: Brandon Foggo, Nanpeng Yu

Comments: To be published in IEEE Transactions on Smart Grid

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

This paper considers the problem of Phase Identification in power distribution systems. In particular, it focuses on improving supervised learning accuracies by focusing on exploiting some of the problem's information theoretic properties. This focus, along with recent advances in Information Theoretic Machine Learning (ITML), helps us to create two new techniques. The first transforms a bound on information losses into a data selection technique. This is important because phase identification data labels are difficult to obtain in practice. The second interprets the properties of distribution systems in the terms of ITML. This allows us to obtain an improvement in the representation learned by any classifier applied to the problem. We tested these two techniques experimentally on real datasets and have found that they yield phenomenal performance in every case. In the most extreme case, they improve phase identification accuracy from $51.7\%$ to $97.3\%$. Furthermore, since many problems share the physical properties of phase identification exploited in this paper, the techniques can be applied to a wide range of similar problems.

[12]  arXiv:1911.01485 (cross-list from cs.CL) [pdf, other]

Title: Assessing Social and Intersectional Biases in Contextualized Word Representations

Authors: Yi Chern Tan, L. Elisa Celis

Comments: NeurIPS 2019

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG); Machine Learning (stat.ML)

Social bias in machine learning has drawn significant attention, with work ranging from demonstrations of bias in a multitude of applications, curating definitions of fairness for different contexts, to developing algorithms to mitigate bias. In natural language processing, gender bias has been shown to exist in context-free word embeddings. Recently, contextual word representations have outperformed word embeddings in several downstream NLP tasks. These word representations are conditioned on their context within a sentence, and can also be used to encode the entire sentence. In this paper, we analyze the extent to which state-of-the-art models for contextual word representations, such as BERT and GPT-2, encode biases with respect to gender, race, and intersectional identities. Towards this, we propose assessing bias at the contextual word level. This novel approach captures the contextual effects of bias missing in context-free word embeddings, yet avoids confounding effects that underestimate bias at the sentence encoding level. We demonstrate evidence of bias at the corpus level, find varying evidence of bias in embedding association tests, show in particular that racial bias is strongly encoded in contextual word models, and observe that bias effects for intersectional minorities are exacerbated beyond their constituent minority identities. Further, evaluating bias effects at the contextual word level captures biases that are not captured at the sentence level, confirming the need for our novel approach.

[13]  arXiv:1911.01486 (cross-list from cs.LG) [pdf, other]

Title: Probabilistic Super-Resolution of Solar Magnetograms: Generating Many Explanations and Measuring Uncertainties

Authors: Xavier Gitiaux, Shane A. Maloney, Anna Jungbluth, Carl Shneider, Paul J. Wright, Atılım Güneş Baydin, Michel Deudon, Yarin Gal, Alfredo Kalaitzis, Andrés Muñoz-Jaramillo

Subjects: Machine Learning (cs.LG); Solar and Stellar Astrophysics (astro-ph.SR); Image and Video Processing (eess.IV); Machine Learning (stat.ML)

Machine learning techniques have been successfully applied to super-resolution tasks on natural images where visually pleasing results are sufficient. However in many scientific domains this is not adequate and estimations of errors and uncertainties are crucial. To address this issue we propose a Bayesian framework that decomposes uncertainties into epistemic and aleatoric uncertainties. We test the validity of our approach by super-resolving images of the Sun's magnetic field and by generating maps measuring the range of possible high resolution explanations compatible with a given low resolution magnetogram.

[14]  arXiv:1911.01496 (cross-list from gr-qc) [pdf, other]

Title: Predicting the properties of black holes merger remnants with Deep Neural Networks

Authors: Leïla Haegel, Sascha Husa

Comments: 14 pages, 3 figures, submitted to CQG

Subjects: General Relativity and Quantum Cosmology (gr-qc); Machine Learning (stat.ML)

We present the first estimation of the mass and spin of Kerr black holes resulting from the coalescence of binary black holes using a deep neural network. The network is trained on the full publicly available catalog of numerical simulations of gravitational waves emission by binary black hole systems. The network prediction for non-precessing binaries as well as precessing binaries is compared with existing fits in the LIGO-Virgo software package when existing. For the non-precessing case, the absolute error distribution has a root mean square error of $2.6 \cdot 10^{-3}$ for the final mass (twice lower than the existing fits) and $3 \cdot 10^{-3}$ for the final spin (similarly to the existing fits). We also estimate of the final mass in the precessing case, where we obtain a RMSE of $1 \cdot 10^{-3}$ of the absolute error distribution. It is $8 \cdot 10^{-3}$ when predicting the spin of the black hole resulting from a precessing binary, against $1.1 \cdot 10^{-2}$ for the existing fits.

[15]  arXiv:1911.01509 (cross-list from cs.LG) [pdf, ps, other]

Title: Understanding racial bias in health using the Medical Expenditure Panel Survey data

Authors: Moninder Singh, Karthikeyan Natesan Ramamurthy

Comments: 8 pages, 8 tables

Subjects: Machine Learning (cs.LG); Computers and Society (cs.CY); Machine Learning (stat.ML)

Over the years, several studies have demonstrated that there exist significant disparities in health indicators in the United States population across various groups. Healthcare expense is used as a proxy for health in algorithms that drive healthcare systems and this exacerbates the existing bias. In this work, we focus on the presence of racial bias in health indicators in the publicly available, and nationally representative Medical Expenditure Panel Survey (MEPS) data. We show that predictive models for care management trained using this data inherit this bias. Finally, we demonstrate that this inherited bias can be reduced significantly using simple mitigation techniques.

[16]  arXiv:1911.01522 (cross-list from math.OC) [pdf, other]

Title: Generalized Self-concordant Hessian-barrier algorithms

Authors: Pavel Dvurechensky, Mathias Staudigl, César A. Uribe

Comments: Submitted to MOR

Subjects: Optimization and Control (math.OC); Computational Complexity (cs.CC); Machine Learning (stat.ML)

Many problems in statistical learning, imaging, and computer vision involve the optimization of a non-convex objective function with singularities at the boundary of the feasible set. For such challenging instances, we develop a new interior-point technique building on the Hessian-barrier algorithm recently introduced in Bomze, Mertikopoulos, Schachinger and Staudigl, [SIAM J. Opt. 2019 29(3), pp. 2100-2127], where the Riemannian metric is induced by a generalized self-concordant function. This class of functions is sufficiently general to include most of the commonly used barrier functions in the literature of interior point methods. We prove global convergence to an approximate stationary point of the method, and in cases where the feasible set admits an easily computable self-concordant barrier, we verify worst-case optimal iteration complexity of the method. Applications in non-convex statistical estimation and $L^{p}$-minimization are discussed to given the efficiency of the method.

[17]  arXiv:1911.01525 (cross-list from math.ST) [pdf, other]

Title: Statistical Inference in Mean-Field Variational Bayes

Authors: Wei Han, Yun Yang

Subjects: Statistics Theory (math.ST); Applications (stat.AP); Computation (stat.CO); Methodology (stat.ME); Machine Learning (stat.ML)

We conduct non-asymptotic analysis on the mean-field variational inference for approximating posterior distributions in complex Bayesian models that may involve latent variables. We show that the mean-field approximation to the posterior can be well-approximated relative to the Kullback-Leibler divergence discrepancy measure by a normal distribution whose center is the maximum likelihood estimator (MLE). In particular, our results imply that the center of the mean-field approximation matches the MLE up to higher-order terms and there is essentially no loss of efficiency in using it as a point estimator for the parameter in any regular parametric model with latent variables. We also propose a new class of variational weighted likelihood bootstrap (VWLB) methods for quantifying the uncertainty in the mean-field variational inference. The proposed VWLB can be viewed as a new sampling scheme that produces independent samples for approximating the posterior. Comparing with traditional sampling algorithms such Markov Chain Monte Carlo, VWLB can be implemented in parallel and is free of tuning.

[18]  arXiv:1911.01531 (cross-list from eess.SP) [pdf, other]

Title: Real-Time Sensor Anomaly Detection and Recovery in Connected Automated Vehicle Sensors

Authors: Yiyang Wang, Neda Masoud, Anahita Khojandi

Comments: 11 pages, 4 figures. Submitted to IEEE Transactions on Intelligent Transportation Systems

Subjects: Signal Processing (eess.SP); Machine Learning (stat.ML)

In this paper we propose a novel observer-based method to improve the safety and security of connected and automated vehicle (CAV) transportation. The proposed method combines model-based signal filtering and anomaly detection methods. Specifically, we use adaptive extended Kalman filter (AEKF) to smooth sensor readings of a CAV based on a nonlinear car-following model. Using the car-following model the subject vehicle (i.e., the following vehicle) utilizes the leading vehicle's information to detect sensor anomalies by employing previously-trained One Class Support Vector Machine (OCSVM) models. This approach allows the AEKF to estimate the state of a vehicle not only based on the vehicle's location and speed, but also by taking into account the state of the surrounding traffic. A communication time delay factor is considered in the car-following model to make it more suitable for real-world applications. Our experiments show that compared with the AEKF with a traditional $\chi^2$-detector, our proposed method achieves a better anomaly detection performance. We also demonstrate that a larger time delay factor has a negative impact on the overall detection performance.

[19]  arXiv:1911.01544 (cross-list from math.ST) [pdf, other]

Title: The generalization error of max-margin linear classifiers: High-dimensional asymptotics in the overparametrized regime

Authors: Andrea Montanari, Feng Ruan, Youngtak Sohn, Jun Yan

Comments: 52 pages; 9 pdf figures

Subjects: Statistics Theory (math.ST); Machine Learning (stat.ML)

Modern machine learning models are often so complex that they achieve vanishing classification error on the training set. Max-margin linear classifiers are among the simplest classification methods that have zero training error (with linearly separable data). Despite this simplicity, their high-dimensional behavior is not yet completely understood. We assume to be given i.i.d. data $(y_i,{\boldsymbol x}_i)$, $i\le n$ with ${\boldsymbol x}_i\sim {\sf N}({\boldsymbol 0},{\boldsymbol \Sigma})$ a $p$-dimensional Gaussian feature vector, and $y_i \in\{+1,-1\}$ a label whose distribution depends on a linear combination of the covariates $\langle {\boldsymbol \theta}_*,{\boldsymbol x}_i\rangle$. We consider the proportional asymptotics $n,p\to\infty$ with $p/n\to \psi$, and derive exact expressions for the limiting prediction error. Our asymptotic results match simulations already when $n,p$ are of the order of a few hundreds.
We explore several choices for the the pair $({\boldsymbol \theta}_*,{\boldsymbol \Sigma})$, and show that the resulting generalization curve (test error error as a function of the overparametrization ratio $\psi=p/n$) is qualitatively different, depending on this choice. In particular we consider a specific structure of $({\boldsymbol \theta}_*,{\boldsymbol \Sigma})$ that captures the behavior of nonlinear random feature models or, equivalently, two-layers neural networks with random first layer weights. In this case, we observe that the test error is monotone decreasing in the number of parameters. This finding agrees with the recently developed `double descent' phenomenology for overparametrized models.

[20]  arXiv:1911.01545 (cross-list from cs.LG) [pdf, other]

Title: Memory Augmented Recursive Neural Networks

Authors: Forough Arabshahi, Zhichu Lu, Sameer Singh, Animashree Anandkumar

Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)

Recursive neural networks have shown an impressive performance for modeling compositional data compared to their recurrent counterparts. Although recursive neural networks are better at capturing long range dependencies, their generalization performance starts to decay as the test data becomes more compositional and potentially deeper than the training data. In this paper, we present memory-augmented recursive neural networks to address this generalization performance loss on deeper data points. We augment Tree-LSTMs with an external memory, namely neural stacks. We define soft push and pop operations for filling and emptying the memory to ensure that the networks remain end-to-end differentiable. In order to assess the effectiveness of the external memory, we evaluate our model on a neural programming task introduced in the literature called equation verification. Our results indicate that augmenting recursive neural networks with external memory consistently improves the generalization performance on deeper data points compared to the state-of-the-art Tree-LSTM by up to 10%.

[21]  arXiv:1911.01559 (cross-list from cs.LG) [pdf, other]

Title: The Tale of Evil Twins: Adversarial Inputs versus Backdoored Models

Authors: Ren Pang, Xinyang Zhang, Shouling Ji, Yevgeniy Vorobeychik, Xiaopu Luo, Ting Wang

Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)

Despite their tremendous success in a wide range of applications, deep neural network (DNN) models are inherently vulnerable to two types of malicious manipulations: adversarial inputs, which are crafted samples that deceive target DNNs, and backdoored models, which are forged DNNs that misbehave on trigger-embedded inputs. While prior work has intensively studied the two attack vectors in parallel, there is still a lack of understanding about their fundamental connection, which is critical for assessing the holistic vulnerability of DNNs deployed in realistic settings.
In this paper, we bridge this gap by conducting the first systematic study of the two attack vectors within a unified framework. More specifically, (i) we develop a new attack model that integrates both adversarial inputs and backdoored models; (ii) with both analytical and empirical evidence, we reveal that there exists an intricate "mutual reinforcement" effect between the two attack vectors; (iii) we demonstrate that this effect enables a large spectrum for the adversary to optimize the attack strategies, such as maximizing attack evasiveness with respect to various defenses and designing trigger patterns satisfying multiple desiderata; (v) finally, we discuss potential countermeasures against this unified attack and their technical challenges, which lead to several promising research directions.

[22]  arXiv:1911.01575 (cross-list from math.OC) [pdf, other]

Title: Importance Sampling via Local Sensitivity

Authors: Anant Raj, Cameron Musco, Lester Mackey

Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)

Given a loss function $F:\mathcal{X} \rightarrow \mathbb{R}^+$ that can be written as the sum of losses over a large set of inputs $a_1,\ldots, a_n$, it is often desirable to approximate $F$ by subsampling the input points. Strong theoretical guarantees require taking into account the importance of each point, measured by how much its individual loss contributes to $F(x)$. Maximizing this importance over all $x \in \mathcal{X}$ yields the \emph{sensitivity score} of $a_i$. Sampling with probabilities proportional to these scores gives strong provable guarantees, allowing one to approximately minimize of $F$ using just the subsampled points. Unfortunately, sensitivity sampling is difficult to apply since 1) it is unclear how to efficiently compute the sensitivity scores and 2) the sample size required is often too large to be useful. We propose overcoming both obstacles by introducing the \emph{local sensitivity}, which measures data point importance in a ball around some center $x_0$. We show that the local sensitivity can be efficiently estimated using the \emph{leverage scores} of a quadratic approximation to $F$, and that the sample size required to approximate $F$ around $x_0$ can be bounded. We propose employing local sensitivity sampling in an iterative optimization method and illustrate its usefulness by analyzing its convergence when $F$ is smooth and convex.

[23]  arXiv:1911.01641 (cross-list from cs.LG) [pdf, other]

Title: New Potential-Based Bounds for Prediction with Expert Advice

Authors: Vladimir A. Kobzar, Robert V. Kohn, Zhilei Wang

Subjects: Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT); Analysis of PDEs (math.AP); Optimization and Control (math.OC); Machine Learning (stat.ML)

This work addresses the classic machine learning problem of online prediction with expert advice. We consider the finite-horizon version of this zero-sum, two-person game. Using verification arguments from optimal control theory, we view the task of finding better lower and upper bounds on the value of the game (regret) as the problem of finding better sub- and supersolutions of certain partial differential equations (PDEs). These sub- and supersolutions serve as the potentials for player and adversary strategies, which lead to the corresponding bounds. Our techniques extend in a nonasymptotic setting the recent work of Drenska and Kohn (J. Nonlinear Sci. 2019), which showed that the asymptotically optimal value function is the unique solution of an associated nonlinear PDE. To get explicit bounds, we use closed-form solutions of specific PDEs. Our bounds hold for any fixed number of experts and any time-horizon $T$; in certain regimes (which we identify) they improve upon the previous state-of-the-art. For up to three experts, our bounds provide the asymptotically optimal leading order term. Therefore, we provide a continuum perspective on recent work on optimal strategies for the case of $N \leq 3$ experts. We expect that our framework could be used to systematize and advance theory and applications of online learning in other settings as well.

[24]  arXiv:1911.01649 (cross-list from cs.LG) [pdf]

Title: Study of Constrained Network Structures for WGANs on Numeric Data Generation

Authors: Wei Wang, Chuang Wang, Tao Cui, Yue Li

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Some recent studies have suggested using GANs for numeric data generation such as to generate data for completing the imbalanced numeric data. Considering the significant difference between the dimensions of the numeric data and images, as well as the strong correlations between features of numeric data, the conventional GANs normally face an overfitting problem, consequently leads to an ill-conditioning problem in generating numeric and structured data. This paper studies the constrained network structures between generator G and discriminator D in WGAN, designs several structures including isomorphic, mirror and self-symmetric structures. We evaluates the performances of the constrained WGANs in data augmentations, taking the non-constrained GANs and WGANs as the baselines. Experiments prove the constrained structures have been improved in 17/20 groups of experiments. In twenty experiments on four UCI Machine Learning Repository datasets, Australian Credit Approval data, German Credit data, Pima Indians Diabetes data and SPECT heart data facing five conventional classifiers. Especially, Isomorphic WGAN is the best in 15/20 experiments. Finally, we theoretically proves that the effectiveness of constrained structures by the directed graphic model (DGM) analysis.

[25]  arXiv:1911.01654 (cross-list from cs.LG) [pdf, other]

Title: Detecting Point Outliers Using Prune-based Outlier Factor (PLOF)

Authors: Kasra Babaei, ZhiYuan Chen, Tomas Maul

Comments: Accepted by "The 4th International Conference on Computing, Mathematics and Statistics 2019 (iCMS2019)"

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Outlier detection (also known as anomaly detection or deviation detection) is a process of detecting data points in which their patterns deviate significantly from others. It is common to have outliers in industry applications, which could be generated by different causes such as human error, fraudulent activities, or system failure. Recently, density-based methods have shown promising results, particularly among which Local Outlier Factor (LOF) is arguably dominating. However, one of the major drawbacks of LOF is that it is computationally expensive. Motivated by the mentioned problem, this research presents a novel pruning-based procedure in which the execution time of LOF is reduced while the performance is maintained. A novel Prune-based Local Outlier Factor (PLOF) approach is proposed, in which prior to employing LOF, outlierness of each data instance is measured. Next, based on a threshold, data instances that require further investigation are separated and LOF score is only computed for these points. Extensive experiments have been conducted and results are promising. Comparison experiments with the original LOF and two state-of-the-art variants of LOF have shown that PLOF produces higher accuracy and precision while reducing execution time.

[26]  arXiv:1911.01658 (cross-list from cs.LG) [pdf, other]

Title: Joint Ranking SVM and Binary Relevance with Robust Low-Rank Learning for Multi-Label Classification

Authors: Guoqiang Wu, Ruobing Zheng, Yingjie Tian, Dalian Liu

Comments: 57 pages, 5 figures, to be published in the journal of Neural Networks

Journal-ref: Neural Networks, 2020, 122: 24-39

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Multi-label classification studies the task where each example belongs to multiple labels simultaneously. As a representative method, Ranking Support Vector Machine (Rank-SVM) aims to minimize the Ranking Loss and can also mitigate the negative influence of the class-imbalance issue. However, due to its stacking-style way for thresholding, it may suffer error accumulation and thus reduces the final classification performance. Binary Relevance (BR) is another typical method, which aims to minimize the Hamming Loss and only needs one-step learning. Nevertheless, it might have the class-imbalance issue and does not take into account label correlations. To address the above issues, we propose a novel multi-label classification model, which joints Ranking support vector machine and Binary Relevance with robust Low-rank learning (RBRL). RBRL inherits the ranking loss minimization advantages of Rank-SVM, and thus overcomes the disadvantages of BR suffering the class-imbalance issue and ignoring the label correlations. Meanwhile, it utilizes the hamming loss minimization and one-step learning advantages of BR, and thus tackles the disadvantages of Rank-SVM including another thresholding learning step. Besides, a low-rank constraint is utilized to further exploit high-order label correlations under the assumption of low dimensional label space. Furthermore, to achieve nonlinear multi-label classifiers, we derive the kernelization RBRL. Two accelerated proximal gradient methods (APG) are used to solve the optimization problems efficiently. Extensive comparative experiments with several state-of-the-art methods illustrate a highly competitive or superior performance of our method RBRL.

[27]  arXiv:1911.01679 (cross-list from cs.LG) [pdf, other]

Title: Apprenticeship Learning via Frank-Wolfe

Authors: Tom Zahavy, Alon Cohen, Haim Kaplan, Yishay Mansour

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

We consider the applications of the Frank-Wolfe (FW) algorithm for Apprenticeship Learning (AL). In this setting, there is a Markov Decision Process (MDP), but the reward function is not given explicitly. Instead, there is an expert that acts according to some policy, and the goal is to find a policy whose feature expectations are closest to those of the expert policy. We formulate this problem as finding the projection of the feature expectations of the expert on the feature expectations polytope -- the convex hull of the feature expectations of all the deterministic policies in the MDP. We show that this formulation is equivalent to the AL objective and that solving this problem using the FW algorithm is equivalent to the most known AL algorithm, the projection method of Abbeel andNg (2004). This insight allows us to analyze AL with tools from the convex optimization literature and to derive tighter bounds on AL. Specifically, we show that a variation of the FW method that is based on taking "away steps" achieves a linear rate of convergence when applied to AL. We also show experimentally that this version outperforms the FW baseline. To the best of our knowledge, this is the first work that shows linear convergence rates for AL.

[28]  arXiv:1911.01694 (cross-list from cs.LG) [pdf, ps, other]

Title: Bounds for the Number of Tests in Non-Adaptive Randomized Algorithms for Group Testing

Authors: Nader H. Bshouty, George Haddad, Catherine A. Haddad-Zaknoon

Subjects: Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)

We study the group testing problem with non-adaptive randomized algorithms. Several models have been discussed in the literature to determine how to randomly choose the tests. For a model ${\cal M}$, let $m_{\cal M}(n,d)$ be the minimum number of tests required to detect at most $d$ defectives within $n$ items, with success probability at least $1-\delta$, for some constant $\delta$. In this paper, we study the measures $$c_{\cal M}(d)=\lim_{n\to \infty} \frac{m_{\cal M}(n,d)}{\ln n} \mbox{ and } c_{\cal M}=\lim_{d\to \infty} \frac{c_{\cal M}(d)}{d}.$$
In the literature, the analyses of such models only give upper bounds for $c_{\cal M}(d)$ and $c_{\cal M}$, and for some of them, the bounds are not tight. We give new analyses that yield tight bounds for $c_{\cal M}(d)$ and $c_{\cal M}$ for all the known models~${\cal M}$.

[29]  arXiv:1911.01695 (cross-list from cs.LG) [pdf, other]

Title: Towards Optimal and Efficient Best Arm Identification in Linear Bandits

Authors: Mohammadi Zaki, Aditya Gopalan, Avinash Mohan

Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)

We give a new algorithm for best arm identification in linearly parameterised bandits in the fixed confidence setting. The algorithm generalises the well-known LUCB algorithm of Kalyanakrishnan et al. (2012) by playing an arm which minimises a suitable notion of geometric overlap of the statistical confidence set for the unknown parameter, and is fully adaptive and computationally efficient as compared to several state-of-the methods. We theoretically analyse the sample complexity of the algorithm for problems with two and three arms, showing optimality in many cases. Numerical results indicate favourable performance over other algorithms with which we compare.

[30]  arXiv:1911.01700 (cross-list from q-fin.CP) [pdf, other]

Title: Deep Hedging: Learning to Simulate Equity Option Markets

Authors: Magnus Wiese, Lianjun Bai, Ben Wood, Hans Buehler

Subjects: Computational Finance (q-fin.CP); Machine Learning (cs.LG); Mathematical Finance (q-fin.MF); Statistical Finance (q-fin.ST); Machine Learning (stat.ML)

We construct realistic equity option market simulators based on generative adversarial networks (GANs). We consider recurrent and temporal convolutional architectures, and assess the impact of state compression. Option market simulators are highly relevant because they allow us to extend the limited real-world data sets available for the training and evaluation of option trading strategies. We show that network-based generators outperform classical methods on a range of benchmark metrics, and adversarial training achieves the best performance. Our work demonstrates for the first time that GANs can be successfully applied to the task of generating multivariate financial time series.

[31]  arXiv:1911.01702 (cross-list from cs.LG) [pdf, other]

Title: DocParser: Hierarchical Structure Parsing of Document Renderings

Authors: Johannes Rausch, Octavio Martinez, Fabian Bissig, Ce Zhang, Stefan Feuerriegel

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)

Translating document renderings (e.g. PDFs, scans) into hierarchical structures is extensively demanded in the daily routines of many real-world applications, and is often a prerequisite step of many downstream NLP tasks. Earlier attempts focused on different but simpler tasks such as the detection of table or cell locations within documents; however, a holistic, principled approach to inferring the complete hierarchical structure in documents is missing. As a remedy, we developed "DocParser": an end-to-end system for parsing the complete document structure - including all text elements, figures, tables, and table cell structures. To the best of our knowledge, DocParser is the first system that derives the full hierarchical document compositions. Given the complexity of the task, annotating appropriate datasets is costly. Therefore, our second contribution is to provide a dataset for evaluating hierarchical document structure parsing. Our third contribution is to propose a scalable learning framework for settings where domain-specific data is scarce, which we address by a novel approach to weak supervision. Our computational experiments confirm the effectiveness of our proposed weak supervision: Compared to the baseline without weak supervision, it improves the mean average precision for detecting document entities by 37.1%. When classifying hierarchical relations between entity pairs, it improves the F1 score by 27.6%.

[32]  arXiv:1911.01705 (cross-list from cs.LG) [pdf, other]

Title: A GMM based algorithm to generate point-cloud and its application to neuroimaging

Authors: Liu Yang, Rudrasis Chakraborty

Subjects: Machine Learning (cs.LG); Image and Video Processing (eess.IV); Machine Learning (stat.ML)

Recent years have witnessed the emergence of 3D medical imaging techniques with the development of 3D sensors and technology. Due to the presence of noise in image acquisition, registration researchers focused on an alternative way to represent medical images. An alternative way to analyze medical imaging is by understanding the 3D shapes represented in terms of point-cloud. Though in the medical imaging community, 3D point-cloud processing is not a ``go-to'' choice, it is a ``natural'' way to capture 3D shapes. However, as the number of samples for medical images are small, researchers have used pre-trained models to fine-tune on medical images. Furthermore, due to different modality in medical images, standard generative models can not be used to generate new samples of medical images. In this work, we use the advantage of point-cloud representation of 3D structures of medical images and propose a Gaussian mixture model-based generation scheme. Our proposed method is robust to outliers. Experimental validation has been performed to show that the proposed scheme can generate new 3D structures using interpolation techniques, i.e., given two 3D structures represented as point-clouds, we can generate point-clouds in between. We have also generated new point-clouds for subjects with and without dementia and show that the generated samples are indeed closely matched to the respective training samples from the same class.

[33]  arXiv:1911.01731 (cross-list from cs.LG) [pdf, other]

Title: GraphAIR: Graph Representation Learning with Neighborhood Aggregation and Interaction

Authors: Fenyu Hu, Yanqiao Zhu, Shu Wu, Weiran Huang, Liang Wang, Tieniu Tan

Comments: 16 pages

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Graph representation learning is of paramount importance for a variety of graph analytical tasks, ranging from node classification to community detection. Recently, graph convolutional networks (GCNs) have been successfully applied for graph representation learning. These GCNs generate node representation by aggregating features from the neighborhoods, which follows the "neighborhood aggregation" scheme. In spite of having achieved promising performance on various tasks, existing GCN-based models have difficulty in well capturing complicated non-linearity of graph data. In this paper, we first theoretically prove that coefficients of the neighborhood interacting terms are relatively small in current models, which explains why GCNs barely outperforms linear models. Then, in order to better capture the complicated non-linearity of graph data, we present a novel GraphAIR framework which models the neighborhood interaction in addition to neighborhood aggregation. Comprehensive experiments conducted on benchmark tasks including node classification and link prediction using public datasets demonstrate the effectiveness of the proposed method over the state-of-the-art methods.

[34]  arXiv:1911.01738 (cross-list from eess.IV) [pdf, other]

Title: Weakly Supervised Fine Tuning Approach for Brain Tumor Segmentation Problem

Authors: Sergey Pavlov, Alexey Artemov, Maksim Sharaev, Alexander Bernstein, Evgeny Burnaev

Comments: Accepted to IEEE International Conference on Machine Learning and Applications (ICMLA 2019)

Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Machine Learning (stat.ML)

Segmentation of tumors in brain MRI images is a challenging task, where most recent methods demand large volumes of data with pixel-level annotations, which are generally costly to obtain. In contrast, image-level annotations, where only the presence of lesion is marked, are generally cheap, generated in far larger volumes compared to pixel-level labels, and contain less labeling noise. In the context of brain tumor segmentation, both pixel-level and image-level annotations are commonly available; thus, a natural question arises whether a segmentation procedure could take advantage of both. In the present work we: 1) propose a learning-based framework that allows simultaneous usage of both pixel- and image-level annotations in MRI images to learn a segmentation model for brain tumor; 2) study the influence of comparative amounts of pixel- and image-level annotations on the quality of brain tumor segmentation; 3) compare our approach to the traditional fully-supervised approach and show that the performance of our method in terms of segmentation quality may be competitive.

[35]  arXiv:1911.01812 (cross-list from cs.LG) [pdf, other]

Title: Enhancing the Privacy of Federated Learning with Sketching

Authors: Zaoxing Liu, Tian Li, Virginia Smith, Vyas Sekar

Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Networking and Internet Architecture (cs.NI); Machine Learning (stat.ML)

In response to growing concerns about user privacy, federated learning has emerged as a promising tool to train statistical models over networks of devices while keeping data localized. Federated learning methods run training tasks directly on user devices and do not share the raw user data with third parties. However, current methods still share model updates, which may contain private information (e.g., one's weight and height), during the training process. Existing efforts that aim to improve the privacy of federated learning make compromises in one or more of the following key areas: performance (particularly communication cost), accuracy, or privacy. To better optimize these trade-offs, we propose that \textit{sketching algorithms} have a unique advantage in that they can provide both privacy and performance benefits while maintaining accuracy. We evaluate the feasibility of sketching-based federated learning with a prototype on three representative learning models. Our initial findings show that it is possible to provide strong privacy guarantees for federated learning without sacrificing performance or accuracy. Our work highlights that there exists a fundamental connection between privacy and communication in distributed settings, and suggests important open problems surrounding the theoretical understanding, methodology, and system design of practical, private federated learning.

[36]  arXiv:1911.01861 (cross-list from cs.LG) [pdf, other]

Title: Biconditional Generative Adversarial Networks for Multiview Learning with Missing Views

Authors: Anastasiia Doinychko, Massih-Reza Amini

Comments: 15 pages, 1 figure

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

In this paper, we present a conditional GAN with two generators and a common discriminator for multiview learning problems where observations have two views, but one of them may be missing for some of the training samples. This is for example the case for multilingual collections where documents are not available in all languages. Some studies tackled this problem by assuming the existence of view generation functions to approximately complete the missing views; for example Machine Translation to translate documents into the missing languages. These functions generally require an external resource to be set and their quality has a direct impact on the performance of the learned multiview classifier over the completed training set. Our proposed approach addresses this problem by jointly learning the missing views and the multiview classifier using a tripartite game with two generators and a discriminator. Each of the generators is associated to one of the views and tries to fool the discriminator by generating the other missing view conditionally on the corresponding observed view. The discriminator then tries to identify if for an observation, one of its views is completed by one of the generators or if both views are completed along with its class. Our results on a subset of Reuters RCV1/RCV2 collections show that the discriminator achieves significant classification performance; and that the generators learn the missing views with high quality without the need of any consequent external resource.

[37]  arXiv:1911.01871 (cross-list from cs.LG) [pdf, ps, other]

Title: On Online Learning in Kernelized Markov Decision Processes

Authors: Sayak Ray Chowdhury, Aditya Gopalan

Comments: arXiv admin note: text overlap with arXiv:1805.08052

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Thompson Sampling (PSRL) philosophies, and work in the general setting of continuous state and action spaces when the true unknown transition dynamics are assumed to have smoothness induced by an appropriate Reproducing Kernel Hilbert Space (RKHS).

[38]  arXiv:1911.01872 (cross-list from hep-ph) [pdf, other]

Title: Interpretability Study on Deep Learning for Jet Physics at the Large Hadron Collider

Authors: Taoli Cheng

Comments: Version for Machine Learning and the Physical Sciences workshop (NeurIPS 2019)

Subjects: High Energy Physics - Phenomenology (hep-ph); Machine Learning (stat.ML)

Using deep neural networks for identifying physics objects at the Large Hadron Collider (LHC) has become a powerful alternative approach in recent years. After successful training of deep neural networks, examining the trained networks not only helps us understand the behaviour of neural networks, but also helps improve the performance of deep learning models through proper interpretation. We take jet tagging problem at the LHC as an example, using recursive neural networks as a starting point, aim at a thorough understanding of the behaviour of the physics-oriented DNNs and the information encoded in the embedding space. We make a comparative study on a series of different jet tagging tasks dominated by different underlying physics. Interesting observations on the latent space are obtained.

[39]  arXiv:1911.01877 (cross-list from eess.IV) [pdf, other]

Title: Out of distribution detection for intra-operative functional imaging

Authors: Tim J. Adler, Leonardo Ayala, Lynton Ardizzone, Hannes G. Kenngott, Anant Vemuri, Beat P. Müller-Stich, Carsten Rother, Ullrich Köthe, Lena Maier-Hein

Comments: The final authenticated version is available online at this https URL

Journal-ref: Proceedings of the First International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2019, and the 8th International Workshop on Clinical Image-Based Procedures, CLIP 2019

Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG); Medical Physics (physics.med-ph); Machine Learning (stat.ML)

Multispectral optical imaging is becoming a key tool in the operating room. Recent research has shown that machine learning algorithms can be used to convert pixel-wise reflectance measurements to tissue parameters, such as oxygenation. However, the accuracy of these algorithms can only be guaranteed if the spectra acquired during surgery match the ones seen during training. It is therefore of great interest to detect so-called out of distribution (OoD) spectra to prevent the algorithm from presenting spurious results. In this paper we present an information theory based approach to OoD detection based on the widely applicable information criterion (WAIC). Our work builds upon recent methodology related to invertible neural networks (INN). Specifically, we make use of an ensemble of INNs as we need their tractable Jacobians in order to compute the WAIC. Comprehensive experiments with in silico, and in vivo multispectral imaging data indicate that our approach is well-suited for OoD detection. Our method could thus be an important step towards reliable functional imaging in the operating room.

[40]  arXiv:1911.01894 (cross-list from cs.LG) [pdf, other]

Title: A Rule for Gradient Estimator Selection, with an Application to Variational Inference

Authors: Tomas Geffner, Justin Domke

Comments: 18 pages, preliminary work

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Stochastic gradient descent (SGD) is the workhorse of modern machine learning. Sometimes, there are many different potential gradient estimators that can be used. When so, choosing the one with the best tradeoff between cost and variance is important. This paper analyzes the convergence rates of SGD as a function of time, rather than iterations. This results in a simple rule to select the estimator that leads to the best optimization convergence guarantee. This choice is the same for different variants of SGD, and with different assumptions about the objective (e.g. convexity or smoothness). Inspired by this principle, we propose a technique to automatically select an estimator when a finite pool of estimators is given. Then, we extend to infinite pools of estimators, where each one is indexed by control variate weights. This is enabled by a reduction to a mixed-integer quadratic program. Empirically, automatically choosing an estimator performs comparably to the best estimator chosen with hindsight.

[41]  arXiv:1911.01914 (cross-list from cs.LG) [pdf, ps, other]

Title: A Comparative Analysis of XGBoost

Authors: Candice Bentéjac, Anna Csörgő, Gonzalo Martínez-Muñoz

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

XGBoost is a scalable ensemble technique based on gradient boosting that has demonstrated to be a reliable and efficient machine learning challenge solver. This work proposes a practical analysis of how this novel technique works in terms of training speed, generalization performance and parameter setup. In addition, a comprehensive comparison between XGBoost, random forests and gradient boosting has been performed using carefully tuned models as well as using the default settings. The results of this comparison may indicate that XGBoost is not necessarily the best choice under all circumstances. Finally an extensive analysis of XGBoost parametrization tuning process is carried out.

[42]  arXiv:1911.01915 (cross-list from cs.LG) [pdf, other]

Title: Scalable Variational Gaussian Processes for Crowdsourcing: Glitch Detection in LIGO

Authors: Pablo Morales-Álvarez, Pablo Ruiz, Scott Coughlin, Rafael Molina, Aggelos K. Katsaggelos

Comments: 16 pages, under review

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); General Relativity and Quantum Cosmology (gr-qc); Machine Learning (stat.ML)

In the last years, crowdsourcing is transforming the way classification training sets are obtained. Instead of relying on a single expert annotator, crowdsourcing shares the labelling effort among a large number of collaborators. For instance, this is being applied to the data acquired by the laureate Laser Interferometer Gravitational Waves Observatory (LIGO), in order to detect glitches which might hinder the identification of true gravitational-waves. The crowdsourcing scenario poses new challenging difficulties, as it deals with different opinions from a heterogeneous group of annotators with unknown degrees of expertise. Probabilistic methods, such as Gaussian Processes (GP), have proven successful in modeling this setting. However, GPs do not scale well to large data sets, which hampers their broad adoption in real practice (in particular at LIGO). This has led to the recent introduction of deep learning based crowdsourcing methods, which have become the state-of-the-art. However, the accurate uncertainty quantification of GPs has been partially sacrificed. This is an important aspect for astrophysicists in LIGO, since a glitch detection system should provide very accurate probability distributions of its predictions. In this work, we leverage the most popular sparse GP approximation to develop a novel GP based crowdsourcing method that factorizes into mini-batches. This makes it able to cope with previously-prohibitive data sets. The approach, which we refer to as Scalable Variational Gaussian Processes for Crowdsourcing (SVGPCR), brings back GP-based methods to the state-of-the-art, and excels at uncertainty quantification. SVGPCR is shown to outperform deep learning based methods and previous probabilistic approaches when applied to the LIGO data. Moreover, its behavior and main properties are carefully analyzed in a controlled experiment based on the MNIST data set.

[43]  arXiv:1911.01916 (cross-list from cs.LG) [pdf, other]

Title: Practical Compositional Fairness: Understanding Fairness in Multi-Task ML Systems

Authors: Xuezhi Wang, Nithum Thain, Anu Sinha, Ed H. Chi, Jilin Chen, Alex Beutel

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Most literature in fairness has focused on improving fairness with respect to one single model or one single objective. However, real-world machine learning systems are usually composed of many different components. Unfortunately, recent research has shown that even if each component is ``fair,'' the overall system can still be ``unfair'' \cite{dwork2018fairness}.
In this paper, we focus on how well fairness composes over multiple components in \emph{real systems}. We consider two recently proposed fairness metrics for rankings: exposure and pairwise ranking accuracy gap. We provide theory that demonstrates a set of conditions under which fairness of individual models does compose. We then present an analytical framework for both understanding whether a system's signals can achieve compositional fairness, and diagnosing which of these signals lowers the overall system's end-to-end fairness the most. Despite previously bleak theoretical results, on multiple data-sets---including a large-scale real-world recommender system---we find that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.

[44]  arXiv:1911.01919 (cross-list from stat.AP) [pdf]

Title: Neural Network Based Parameter Estimation Method for the Pareto/NBD Model

Authors: Shao-Ming Xie

Comments: 35 pages, 6 figures, 14 tables

Subjects: Applications (stat.AP); Machine Learning (stat.ML)

Whether stochastic or parametric, the Pareto/NBD model can only be utilized for an in-sample prediction rather than an out-of-sample prediction. This research thus provides a neural network based extension of the Pareto/NBD model to estimate the out-of-sample parameters, which overrides the estimation burden and the application dilemma of the Pareto/NBD approach. The empirical results indicate that the Pareto/NBD model and neural network algorithms have similar predictability for identifying inactive customers. Even with a strong trend fitting on the customer count of each repeat purchase point, the Pareto/NBD model underestimates repeat purchases at both the individual and aggregate levels. Nonetheless, when embedding the likelihood function of the Pareto/NBD model into the loss function, the proposed parameter estimation method shows extraordinary predictability on repeat purchases at these two levels. Furthermore, the proposed neural network based method is highly efficient and resource-friendly and can be deployed in cloud computing to handle with big data analysis.

[45]  arXiv:1911.01931 (cross-list from cs.LG) [pdf, other]

Title: Online matrix factorization for Markovian data and applications to Network Dictionary Learning

Authors: Hanbaek Lyu, Deana Needell, Laura Balzano

Comments: 32 pages, 12 figures

Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Optimization and Control (math.OC); Probability (math.PR); Machine Learning (stat.ML)

Online Matrix Factorization (OMF) is a fundamental tool for dictionary learning problems, giving an approximate representation of complex data sets in terms of a reduced number of extracted features. Convergence guarantees for most of the OMF algorithms in the literature assume independence between data matrices, and the case of a dependent data stream remains largely unexplored. In this paper, we show that the well-known OMF algorithm for i.i.d. stream of data proposed in \cite{mairal2010online}, in fact converges almost surely to the set of critical points of the expected loss function, even when the data matrices form a Markov chain satisfying a mild mixing condition. Furthermore, we extend the convergence result to the case when we can only approximately solve each step of the optimization problems in the algorithm. For applications, we demonstrate dictionary learning from a sequence of images generated by a Markov Chain Monte Carlo (MCMC) sampler. Lastly, by combining online non-negative matrix factorization and a recent MCMC algorithm for sampling motifs from networks, we propose a novel framework of Network Dictionary Learning, which extracts `network dictionary patches' from a given network in an online manner that encodes main features of the network. We demonstrate this technique on real-world text data.

[46]  arXiv:1911.01933 (cross-list from cs.LG) [pdf, ps, other]

Title: Training Neural Machine Translation (NMT) Models using Tensor Train Decomposition on TensorFlow (T3F)

Authors: Amelia Drew, Alexander Heinecke

Comments: 10 pages, 2 tables

Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Machine Learning (stat.ML)

We implement a Tensor Train layer in the TensorFlow Neural Machine Translation (NMT) model using the t3f library. We perform training runs on the IWSLT English-Vietnamese '15 and WMT German-English '16 datasets with learning rates $\in \{0.0004,0.0008,0.0012\}$, maximum ranks $\in \{2,4,8,16\}$ and a range of core dimensions. We compare against a target BLEU test score of 24.0, obtained by our benchmark run. For the IWSLT English-Vietnamese training, we obtain BLEU test/dev scores of 24.0/21.9 and 24.2/21.9 using core dimensions $(2, 2, 256) \times (2, 2, 512)$ with learning rate 0.0012 and rank distributions $(1,4,4,1)$ and $(1,4,16,1)$ respectively. These runs use 113\% and 397\% of the flops of the benchmark run respectively. We find that, of the parameters surveyed, a higher learning rate and more `rectangular' core dimensions generally produce higher BLEU scores. For the WMT German-English dataset, we obtain BLEU scores of 24.0/23.8 using core dimensions $(4, 4, 128) \times (4, 4, 256)$ with learning rate 0.0012 and rank distribution $(1,2,2,1)$. We discuss the potential for future optimization and application of Tensor Train decomposition to other NMT models.

[47]  arXiv:1911.01944 (cross-list from cs.LG) [pdf, other]

Title: Dynamic Time Warp Convolutional Networks

Authors: Yaniv Shulman

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Where dealing with temporal sequences it is fair to assume that the same kind of deformations that motivated the development of the Dynamic Time Warp algorithm could be relevant also in the calculation of the dot product ("convolution") in a 1-D convolution layer. In this work a method is proposed for aligning the convolution filter and the input where they are locally out of phase utilising an algorithm similar to the Dynamic Time Warp. The proposed method enables embedding a non-parametric warping of temporal sequences for increasing similarity directly in deep networks and can expand on the generalisation capabilities and the capacity of standard 1-D convolution layer where local sequential deformations are present in the input. Experimental results demonstrate the proposed method exceeds or matches the standard 1-D convolution layer in terms of the maximum accuracy achieved on a number of time series classification tasks. In addition the impact of different hyperparameters settings is investigated given different datasets and the results support the conclusions of previous work done in relation to the choice of DTW parameter values. The proposed layer can be freely integrated with other typical layers to compose deep artificial neural networks of an arbitrary architecture that are trained using standard stochastic gradient descent.

Replacements for Wed, 6 Nov 19

[48]  arXiv:1611.06534 (replaced) [pdf, other]

Title: Linear Thompson Sampling Revisited

Authors: Marc Abeille, Alessandro Lazaric

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[49]  arXiv:1805.05071 (replaced) [pdf, other]

Title: KL-UCB-switch: optimal regret bounds for stochastic bandits from both a distribution-dependent and a distribution-free viewpoints

Authors: Aurélien Garivier, Hédi Hadiji, Pierre Menard, Gilles Stoltz

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)

[50]  arXiv:1906.08344 (replaced) [pdf, other]

Title: Multi-resolution Multi-task Gaussian Processes

Authors: Oliver Hamelijnck, Theodoros Damoulas, Kangrui Wang, Mark Girolami

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[51]  arXiv:1910.08880 (replaced) [pdf, other]

Title: Sparse (group) learning with Lipschitz loss functions: a unified analysis

Authors: Antoine Dedieu

Comments: arXiv admin note: text overlap with arXiv:1810.03081

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Other Statistics (stat.OT)

[52]  arXiv:1910.12794 (replaced) [pdf, other]

Title: Stein Variational Gradient Descent With Matrix-Valued Kernels

Authors: Dilin Wang, Ziyang Tang, Chandrajit Bajaj, Qiang Liu

Comments: Neural Information Processing Systems 2019

Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

[53]  arXiv:1605.00252 (replaced) [pdf, other]

Title: Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes

Authors: Peter D. Grünwald, Nishant A. Mehta

Comments: accepted to JMLR pending minor final modifications

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[54]  arXiv:1802.07007 (replaced) [pdf]

Title: Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting

Authors: Zhiyong Cui, Kristian Henrickson, Ruimin Ke, Ziyuan Pu, Yinhai Wang

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[55]  arXiv:1806.10758 (replaced) [pdf, other]

Title: A Benchmark for Interpretability Methods in Deep Neural Networks

Authors: Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim

Comments: In NeurIPS 2019

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)

[56]  arXiv:1811.02667 (replaced) [pdf, other]

Title: Band Selection from Hyperspectral Images Using Attention-based Convolutional Neural Networks

Authors: Pablo Ribalta Lorenzo, Lukasz Tulczyjew, Michal Marcinkiewicz, Jakub Nalepa

Comments: This is an initial draft of the paper submitted to IEEE GEOSCIENCE AND REMOTE SENSING LETTERS

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Machine Learning (stat.ML)

[57]  arXiv:1902.05991 (replaced) [pdf, ps, other]

Title: Information Losses in Neural Classifiers from Sampling

Authors: Brandon Foggo, Nanpeng Yu, Jie Shi, Yuanqi Gao

Comments: To be published in IEEE TNNLS

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[58]  arXiv:1902.08605 (replaced) [pdf, other]

Title: Are Few-Shot Learning Benchmarks too Simple ?

Authors: Gabriel Huang, Hugo Larochelle, Simon Lacoste-Julien

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[59]  arXiv:1902.10089 (replaced) [pdf, other]

Title: Perturbed-History Exploration in Stochastic Multi-Armed Bandits

Authors: Branislav Kveton, Csaba Szepesvari, Mohammad Ghavamzadeh, Craig Boutilier

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[60]  arXiv:1903.00904 (replaced) [pdf, other]

Title: adVAE: a Self-adversarial Variational Autoencoder with Gaussian Anomaly Prior Knowledge for Anomaly Detection

Authors: Xuhong Wang, Ying Du, Shijie Lin, Ping Cui, Yupu Yang

Comments: This is a older version. The latest version has been accept by Knowledge-based Systems. Due to the copyright policy, I can't upload the latest version

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)

[61]  arXiv:1903.11431 (replaced) [pdf, other]

Title: Transform Learning for Magnetic Resonance Image Reconstruction: From Model-based Learning to Building Neural Networks

Authors: Bihan Wen, Saiprasad Ravishankar, Luke Pfister, Yoram Bresler

Comments: Accepted to IEEE Signal Processing Magazine, Special Issue on Computational MRI: Compressed Sensing and Beyond

Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG); Machine Learning (stat.ML)

[62]  arXiv:1905.07833 (replaced) [pdf, other]

Title: Shaping the learning landscape in neural networks around wide flat minima

Authors: Carlo Baldassi, Fabrizio Pittorino, Riccardo Zecchina

Comments: 37 pages (16 main text), 10 figures (7 main text)

Subjects: Machine Learning (cs.LG); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (stat.ML)

[63]  arXiv:1905.10498 (replaced) [pdf, other]

Title: Cold Case: The Lost MNIST Digits

Authors: Chhavi Yadav, Léon Bottou

Comments: Final NeurIPS version

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)

[64]  arXiv:1905.11465 (replaced) [pdf, other]

Title: ADDIS: an adaptive discarding algorithm for online FDR control with conservative nulls

Authors: Jinjin Tian, Aaditya Ramdas

Comments: Accepted to Neurips 2019. Corrected some typos

Subjects: Methodology (stat.ME); Statistics Theory (math.ST); Machine Learning (stat.ML)

[65]  arXiv:1905.11947 (replaced) [pdf, ps, other]

Title: Private Identity Testing for High-Dimensional Distributions

Authors: Clément L. Canonne, Gautam Kamath, Audra McMillan, Jonathan Ullman, Lydia Zakynthinou

Comments: Improved the bounds and the writing

Subjects: Data Structures and Algorithms (cs.DS); Cryptography and Security (cs.CR); Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)

[66]  arXiv:1906.04356 (replaced) [pdf, other]

Title: Ultra Fast Medoid Identification via Correlated Sequential Halving

Authors: Tavor Z. Baharav, David N. Tse

Comments: NeurIPS 2019

Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Information Theory (cs.IT); Machine Learning (stat.ML)

[67]  arXiv:1906.04733 (replaced) [pdf, other]

Title: DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections

Authors: Ofir Nachum, Yinlam Chow, Bo Dai, Lihong Li

Comments: Appearing in NeurIPS 2019 Vancouver, Canada

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)

[68]  arXiv:1906.05433 (replaced) [pdf, other]

Title: Tackling Climate Change with Machine Learning

Authors: David Rolnick, Priya L. Donti, Lynn H. Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, Alexandra Luccioni, Tegan Maharaj, Evan D. Sherwin, S. Karthik Mukkavilli, Konrad P. Kording, Carla Gomes, Andrew Y. Ng, Demis Hassabis, John C. Platt, Felix Creutzig, Jennifer Chayes, Yoshua Bengio

Comments: For additional resources, please visit the website that accompanies this paper: this https URL

Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)

[69]  arXiv:1906.06781 (replaced) [pdf, ps, other]

Title: Characterizing the Exact Behaviors of Temporal Difference Learning Algorithms Using Markov Jump Linear System Theory

Authors: Bin Hu, Usman Ahmed Syed

Comments: To appear in NeurIPS 2019

Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)

[70]  arXiv:1906.08253 (replaced) [pdf, other]

Title: When to Trust Your Model: Model-Based Policy Optimization

Authors: Michael Janner, Justin Fu, Marvin Zhang, Sergey Levine

Comments: NeurIPS 2019. Code at this https URL, project page at: this https URL

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)

[71]  arXiv:1906.08283 (replaced) [pdf, other]

Title: Minimum Stein Discrepancy Estimators

Authors: Alessandro Barp, Francois-Xavier Briol, Andrew B. Duncan, Mark Girolami, Lester Mackey

Comments: Accepted for publication at NeurIPS 2019

Subjects: Statistics Theory (math.ST); Machine Learning (cs.LG); Methodology (stat.ME); Machine Learning (stat.ML)

[72]  arXiv:1906.09613 (replaced) [pdf, other]

Title: The Cost of a Reductions Approach to Private Fair Optimization

Authors: Daniel Alabi

Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)

[73]  arXiv:1906.10816 (replaced) [pdf, other]

Title: Program Synthesis and Semantic Parsing with Learned Code Idioms

Authors: Richard Shin, Miltiadis Allamanis, Marc Brockschmidt, Oleksandr Polozov

Comments: 33rd Conference on Neural Information Processing Systems (NeurIPS) 2019. 13 pages total, 9 pages of main text

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Programming Languages (cs.PL); Machine Learning (stat.ML)

[74]  arXiv:1907.01894 (replaced) [pdf, other]

Title: A Bayesian Hierarchical Model for Criminal Investigations

Authors: F.O.Bunnin, J.Q.Smith

Comments: 57 pages, 20 figures, 8 tables

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Applications (stat.AP); Machine Learning (stat.ML)

[75]  arXiv:1907.02114 (replaced) [pdf, ps, other]

Title: Maximum Expected Hitting Cost of a Markov Decision Process and Informativeness of Rewards

Authors: Falcon Z. Dai, Matthew R. Walter

Comments: Minor post-review revision. Main paper with appendix. To appear at NeurIPS 2019

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[76]  arXiv:1907.02544 (replaced) [pdf, other]

Title: Large Scale Adversarial Representation Learning

Authors: Jeff Donahue, Karen Simonyan

Comments: 32 pages. In proceedings of NeurIPS 2019. This is the camera-ready version of the paper, with supplementary material included as appendices

Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Machine Learning (stat.ML)

[77]  arXiv:1907.02690 (replaced) [pdf, other]

Title: Twin Auxiliary Classifiers GAN

Authors: Mingming Gong, Yanwu Xu, Chunyuan Li, Kun Zhang, Kayhan Batmanghelich

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[78]  arXiv:1907.05012 (replaced) [pdf, other]

Title: Making AI Forget You: Data Deletion in Machine Learning

Authors: Antonio Ginart, Melody Y. Guan, Gregory Valiant, James Zou

Comments: To appear in NeurIPS 2019

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[79]  arXiv:1907.09701 (replaced) [pdf, other]

Title: Benchmarking Attribution Methods with Relative Feature Importance

Authors: Mengjiao Yang, Been Kim

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[80]  arXiv:1908.03109 (replaced) [pdf, other]

Title: FAIRY: A Framework for Understanding Relationships between Users' Actions and their Social Feeds

Authors: Azin Ghazimatin, Rishiraj Saha Roy, Gerhard Weikum

Comments: WSDM 2019

Journal-ref: WSDM 2019

Subjects: Social and Information Networks (cs.SI); Machine Learning (cs.LG); Machine Learning (stat.ML)

[81]  arXiv:1909.05024 (replaced) [pdf, other]

Title: Learning to Propagate for Graph Meta-Learning

Authors: Lu Liu, Tianyi Zhou, Guodong Long, Jing Jiang, Chengqi Zhang

Comments: Accepted to NeurIPS 2019, code at this https URL, slides at this https URL, Poster at this https URL

Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)

[82]  arXiv:1909.06588 (replaced) [pdf, ps, other]

Title: Branch and Bound for Piecewise Linear Neural Network Verification

Authors: Rudy Bunel, Jingyue Lu, Ilker Turkaslan, Philip H.S. Torr, Pushmeet Kohli, M. Pawan Kumar

Subjects: Machine Learning (cs.LG); Logic in Computer Science (cs.LO); Machine Learning (stat.ML)

[83]  arXiv:1909.13563 (replaced) [pdf, other]

Title: Training-Free Artificial Neural Networks

Authors: Nikolaos P. Bakas, Savvas Chatzichristofis

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[84]  arXiv:1910.02100 (replaced) [pdf, other]

Title: Social Learning in Multi Agent Multi Armed Bandits

Authors: Abishek Sankararaman, Ayalvadi Ganesh, Sanjay Shakkottai

Comments: Minor Corrections from before

Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Networking and Internet Architecture (cs.NI); Social and Information Networks (cs.SI); Probability (math.PR); Machine Learning (stat.ML)

[85]  arXiv:1910.02366 (replaced) [pdf, other]

Title: Splitting Steepest Descent for Growing Neural Architectures

Authors: Qiang Liu, Lemeng Wu, Dilin Wang

Comments: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)

Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)

[86]  arXiv:1910.05471 (replaced) [pdf, other]

Title: Efficient Inference and Exploration for Reinforcement Learning

Authors: YI Zhu, Jing Dong, Henry Lam

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

[87]  arXiv:1910.10873 (replaced) [pdf, ps, other]

Title: Minimax Regret of Switching-Constrained Online Convex Optimization: No Phase Transition

Authors: Lin Chen, Qian Yu, Hannah Lawrence, Amin Karbasi

Comments: First two authors contributed equally

Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)

[88]  arXiv:1910.12159 (replaced) [pdf]

Title: Classification of Neurodevelopmental Age in Normal Infants Using 3D-CNN based on Brain MRI

Authors: Mahdieh Shabanian, Eugene C. Eckstein, Hao Chen, John P. DeVincenzo

Comments: 6 pages, 11 figures, 6 tables

Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG); Machine Learning (stat.ML)

[89]  arXiv:1911.00405 (replaced) [pdf, other]

Title: Training Neural Networks for Likelihood/Density Ratio Estimation

Authors: George V. Moustakides, Kalliopi Basioti

Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Machine Learning (stat.ML)

[90]  arXiv:1911.00922 (replaced) [pdf, ps, other]

Title: Variable Grouping Based Bayesian Additive Regression Tree

Authors: Yuhao Su, Jie Ding

Comments: 5 pages, 3 tables

Subjects: Machine Learning (cs.LG); Signal Processing (eess.SP); Methodology (stat.ME); Machine Learning (stat.ML)

• New submissions
• Cross-lists
• Replacements