Handbook of Big Data Analytics

Preface

Contents

Part I Overview

1 Statistics, Statisticians, and the Internet of Things

1.1 Introduction

1.1.1 The Internet of Things

1.1.2 What Is Big Data in an Internet of Things?

1.1.3 Building Blocks

1.1.4 Ubiquity

1.1.5 Consumer Applications

1.1.6 The Internets of [Infrastructure] Things

1.1.7 Industrial Scenarios

1.2 What Kinds of Statistics Are Needed for Big IoT Data?

1.2.1 Coping with Complexity

1.2.2 Privacy

1.2.3 Traditional Statistics Versus the IoT

1.2.4 A View of the Future of Statistics in an IoT World

1.3 Big Data in the Real World

1.3.1 Skills

1.3.2 Politics

1.3.3 Technique

1.3.4 Traditional Databases

1.3.5 Cognition

1.4 Conclusion

2 Cognitive Data Analysis for Big Data

2.1 Introduction

2.1.1 Big Data

2.1.2 Defining Cognitive Data Analysis

2.1.3 Stages of CDA

2.2 Data Preparation

2.2.1 Natural Language Query

2.2.2 Data Integration

2.2.3 Metadata Discovery

2.2.4 Data Quality Verification

2.2.5 Data Type Detection

2.2.6 Data Lineage

2.3 Automated Modeling

2.3.1 Descriptive Analytics

2.3.2 Predictive Analytics

2.3.3 Starting Points

2.3.4 System Recommendations

2.4 Application of Results

2.4.1 Gaining Insights

2.4.2 Sharing and Collaborating

2.4.3 Deployment

2.5 Use Case

2.6 Conclusion

References

Part II Methodology

3 Statistical Leveraging Methods in Big Data

3.1 Background

3.2 Leveraging Approximation for Least Squares Estimator

3.2.1 Leveraging for Least Squares Approximation

3.2.2 A Matrix Approximation Perspective

3.2.3 The Computation of Leveraging Scores

3.2.4 An Innovative Proposal: Predictor-Length Method

3.2.5 More on Modeling

3.2.6 Statistical Leveraging Algorithms in the Literature: A Summary

3.3 Statistical Properties of Leveraging Estimator

3.3.1 Weighted Leveraging Estimator

3.3.2 Unweighted Leveraging Estimator

3.4 Simulation Study

3.4.1 UNIF and BLEV

3.4.2 BLEV and LEVUNW

3.4.3 BLEV and SLEV

3.4.4 BLEV and PL

3.4.5 SLEV and PL

3.5 Real Data Analysis

3.6 Beyond Linear Regression

3.6.1 Logistic Regression

3.6.2 Time Series Analysis

3.7 Discussion and Conclusion

References

4 Scattered Data and Aggregated Inference

4.1 Introduction

4.2 Problem Formulation

4.2.1 Notations

4.2.2 Review on M-Estimators

4.2.3 Simple Averaging Estimator

4.2.4 One-Step Estimator

4.3 Main Results

4.3.1 Assumptions

4.3.2 Asymptotic Properties and Mean Squared Errors (MSE) Bounds

4.3.3 Under the Presence of Communication Failure

4.4 Numerical Examples

4.4.1 Logistic Regression

4.4.2 Beta Distribution

4.4.3 Beta Distribution with Possibility of Losing Information

4.4.4 Gaussian Distribution with Unknown Mean and Variance

4.5 Discussion on Distributed Statistical Inference

4.6 Other Problems

4.7 Conclusion

References

5 Nonparametric Methods for Big Data Analytics

5.1 Introduction

5.2 Classical Methods for Nonparametric Regression

5.2.1 Additive Models

5.2.2 Generalized Additive Models (GAM)

5.2.3 Smoothing Spline ANOVA (SS-ANOVA)

5.3 High Dimensional Additive Models

5.3.1 COSSO Method

5.3.2 Adaptive COSSO

5.3.3 Linear and Nonlinear Discover (LAND)

5.3.4 Adaptive Group LASSO

5.3.5 Sparse Additive Models (SpAM)

5.3.6 Sparsity-Smoothness Penalty

5.4 Nonparametric Independence Screening (NIS)

References

6 Finding Patterns in Time Series

6.1 Introduction

6.1.1 Regime Descriptors: Local Models

6.1.2 Changepoints

6.1.3 Patterns

6.1.4 Clustering, Classification, and Prediction

6.1.5 Measures of Similarity/Dissimilarity

6.1.6 Outline

6.2 Data Reduction and Changepoints

6.2.1 Piecewise Constant Models

6.2.2 Models with Changing Scales

6.2.3 Trends

6.3 Model Building

6.3.1 Batch Methods

6.3.2 Online Methods

6.4 Model Building: Alternating Trends Smoothing

6.4.1 The Tuning Parameter

6.4.2 Modifications and Extensions

6.5 Bounding Lines

6.6 Patterns

6.6.1 Time Scaling and Junk

6.6.2 Further Data Reduction: Symbolic Representation

6.6.3 Symbolic Trend Patterns (STP)

6.6.4 Patterns in Bounding Lines

6.6.5 Clustering and Classification of Time Series

References

7 Variational Bayes for Hierarchical Mixture Models

7.1 Introduction

7.2 Variational Bayes

7.2.1 Overview of the VB Method

7.2.2 Practicality

7.2.3 Over-Confidence

7.2.4 Simple Two-Component Mixture Model

7.2.5 Marginal Posterior Approximation

7.3 VB for a General Finite Mixture Model

7.3.1 Motivation

7.3.2 The B-LIMMA Model

7.4 Numerical Illustrations

7.4.1 Simulation

7.4.1.1 The B-LIMMA Model

7.4.1.2 A Mixture Model Extended from the LIMMA Model

7.4.1.3 A Mixture Model for Count Data

7.4.2 Real Data Examples

7.4.2.1 APOA1 Data

7.4.2.2 Colon Cancer Data

7.5 Discussion

Appendix: The VB-LEMMA Algorithm

The B-LEMMA Model

Algorithm

The VB-Proteomics Algorithm

The Proteomics Model

Algorithm

References

8 Hypothesis Testing for High-Dimensional Data

8.1 Introduction

8.2 Applications

8.2.1 Testing of Covariance Matrices

8.2.2 Testing of Independence

8.2.3 Analysis of Variance

8.3 Tests Based on L? Norms

8.4 Tests Based on L2 Norms

8.5 Asymptotic Theory

8.5.1 Preamble: i.i.d. Gaussian Data

8.5.2 Rademacher Weighted Differencing

8.5.3 Calculating the Power

8.5.4 An Algorithm with General Testing Functionals

8.6 Numerical Experiments

8.6.1 Test of Mean Vectors

8.6.2 Test of Covariance Matrices

8.6.2.1 Sizes Accuracy

8.6.2.2 Power Curve

8.6.3 A Real Data Application

References

9 High-Dimensional Classification

9.1 Introduction

9.2 LDA, Logistic Regression, and SVMs

9.2.1 LDA

9.2.2 Logistic Regression

9.2.3 The Support Vector Machine

9.3 Lasso and Elastic-Net Penalized SVMs

9.3.1 The 1 SVM

9.3.2 The DrSVM

9.4 Lasso and Elastic-Net Penalized Logistic Regression

9.5 Huberized SVMs

9.6 Concave Penalized Margin-Based Classifiers

9.7 Sparse Discriminant Analysis

9.7.1 Independent Rules

9.7.2 Linear Programming Discriminant Analysis

9.7.3 Direct Sparse Discriminant Analysis

9.8 Sparse Semiparametric Discriminant Analysis

9.9 Sparse Penalized Additive Models for Classification

References

10 Analysis of High-Dimensional Regression Models Using Orthogonal Greedy Algorithms

10.1 Introduction

10.2 Convergence Rates of OGA

10.2.1 Random Regressors

10.2.2 The Fixed Design Case

10.3 The Performance OGA Under General Sparse Conditions

10.3.1 Rates of Convergence

10.3.2 Comparative Studies

10.4 The Performance of OGA in High-Dimensional Time Series Models

References

11 Semi-supervised Smoothing for Large Data Problems

11.1 Introduction

11.2 Semi-supervised Local Kernel Regression

11.2.1 Supervised Kernel Regression

11.2.2 Semi-supervised Kernel Regression with a Latent Response

11.2.3 Adaptive Semi-supervised Kernel Regression

11.2.4 Computational Issues for Large Data

11.3 Optimization Frameworks for Semi-supervised Learning

References

12 Inverse Modeling: A Strategy to Cope with Non-linearity

12.1 Introduction

12.2 SDR and Inverse Modeling

12.2.1 From SIR to PFC

12.2.2 Revisit SDR from an Inverse Modeling Perspective

12.3 Variable Selection

12.3.1 Beyond Sufficient Dimension Reduction: The Necessity of Variable Selection

12.3.2 SIR as a Transformation-Projection Pursuit Problem

12.3.3 COP: Correlation Pursuit

12.3.4 From COP to SIRI

12.3.5 Simulation Study for Variable Selection and SDR Estimation

12.4 Nonparametric Dependence Screening

12.5 Conclusion

References

13 Sufficient Dimension Reduction for Tensor Data

13.1 Curse of Dimensionality

13.2 Sufficient Dimension Reduction

13.3 Tensor Sufficient Dimension Reduction

13.3.1 Tensor Sufficient Dimension Reduction Model

13.3.2 Estimate a Single Direction

13.4 Simulation Studies

13.5 Example

13.6 Discussion

References

14 Compressive Sensing and Sparse Coding

14.1 Leveraging the Sparsity Assumption for Signal Recovery

14.2 From Combinatorial to Convex Optimization

14.3 Dealing with Noisy Measurements

14.4 Other Common Forms and Variations

14.5 The Theory Behind

14.5.1 The Restricted Isometry Property

14.5.2 Guaranteed Signal Recovery

14.5.3 Random Matrix is Good Enough

14.6 Compressive Sensing in Practice

14.6.1 Solving the Compressive Sensing Problem

14.6.2 Sparsifying Basis

14.6.3 Sensing Matrix

14.7 Sparse Coding Overview

14.7.1 Compressive Sensing and Sparse Coding

14.7.1.1 Compressed Domain Feature Extraction

14.7.1.2 Compressed Domain Classification

14.8 Compressive Sensing Extensions

14.8.1 Reconstruction with Additional Information

14.8.2 Compressive Sensing with Distorted Measurements

References

15 Bridging Density Functional Theory and Big Data Analytics with Applications

15.1 Introduction

15.2 Structure of Data Functionals Defined in the DFT Perspectives

15.3 Determinations of Number of Data Groups and the Corresponding Data Boundaries

15.4 Physical Phenomena of the Mixed Data Groups

15.4.1 Physical Structure of the DFT-Based Algorithm

15.4.2 Typical Problem of the Data Clustering:The Fisher's Iris

15.4.3 Tentative Experiments on Dataset of MRI with Brain Tumors

15.5 Conclusion

References

Part III Software

16 Q3-D3-LSA: D3.js and Generalized Vector Space Models for Statistical Computing

16.1 Introduction: From Data to Information

16.1.1 Transparency, Collaboration, and Reproducibility

16.2 Related Work

16.3 Q3-D3 Genesis

16.4 Vector Space Representations

16.4.1 Text to Vector

16.4.2 Weighting Scheme, Similarity, Distance

16.4.3 Shakespeare's Tragedies

16.4.4 Generalized VSM (GVSM)

16.4.4.1 Basic VSM (BVSM)

16.4.4.2 GVSM: Term–Term Correlations

16.4.4.3 GVSM: Latent Semantic Analysis (LSA)

16.4.4.4 Closer Look at the LSA Implementation

16.4.4.5 GVSM Applicability for Big Data

16.5 Methods

16.5.1 Cluster Analysis

16.5.1.1 Partitional Clustering

16.5.1.2 Hierarchical Clustering

16.5.2 Cluster Validation Measures

16.5.2.1 Connectivity

16.5.2.2 Silhouette

16.5.2.3 Dunn Index

16.5.3 Visual Cluster Validation

16.6 Results

16.6.1 Text Preprocessing Results

16.6.2 Sparsity Results

16.6.3 Three Models, Three Methods, Three Measures

16.6.4 LSA Anatomy

16.7 Application

16.8 Outlook

16.8.1 GitHub Mining Infrastructure in R

16.8.2 Future Developments

Appendix

References

17 A Tutorial on Libra: R Package for the Linearized Bregman Algorithm in High-Dimensional Statistics

17.1 Introduction to brownLibra

17.2 Linear Model

17.2.1 Example: Simulation Data

17.2.2 Example: Diabetes Data

17.3 Logistic Model

17.3.1 Binomial Logistic Model

17.3.1.1 Example: Publications of COPSS Award Winners

17.3.1.2 Example: Journey to the West

17.3.2 Multinomial Logistic Model

17.4 Graphical Model

17.4.1 Gaussian Graphical Model

17.4.1.1 Example: Journey to the West

17.4.2 Ising Model

17.4.2.1 Example: Simulation Data

17.4.2.2 Example: Journey to the West

17.4.2.3 Example: Dream of the Red Chamber

17.4.3 Potts Model

17.5 Discussion

References

Part IV Application

18 Functional Data Analysis for Big Data: A Case Study on California Temperature Trends

18.1 Introduction

18.2 Basic Statistics for Functional Data

18.3 Dimension Reduction for Functional Data

18.4 Functional Principal Component Analysis

18.4.1 Smoothing and Interpolation

18.4.2 Sample Size Considerations

18.5 Functional Variance Process

18.6 Functional Data Analysis for Temperature Trends

18.7 Conclusions

References

19 Bayesian Spatiotemporal Modeling for Detecting Neuronal Activation via Functional Magnetic Resonance Imaging

19.1 Introduction

19.1.1 Emotion Processing Data

19.2 Variable Selection in Bayesian Spatiotemporal Models

19.2.1 Bezener et al.'s (2015) Areal Model

19.2.1.1 Posterior Distribution and MCMC Algorithm

19.2.1.2 Starting Values

19.2.1.3 Emotion Processing Data

19.2.2 Musgrove et al.'s (2015) Areal Model

19.2.2.1 Partitioning the Image

19.2.2.2 Spatial Bayesian Variable Selection with Temporal Correlation

19.2.2.3 Sparse SGLMM Prior

19.2.2.4 Posterior Computation and Inference

19.2.2.5 Emotion Processing Data

19.2.3 Activation Maps for Emotion Processing Data

19.3 Discussion

References

20 Construction of Tight Frames on Graphs and Application to Denoising

20.1 Introduction

20.1.1 Motivation

20.1.2 Relation to Previous Work

20.2 Notation and Basics

20.2.1 Setting

20.2.2 Frames

20.2.3 Neighborhood Graphs

20.2.4 Spectral Graph Theory

20.3 Construction and Properties

20.3.1 Construction of a Tight Graph Frame

20.3.2 Spatial Localization

20.4 Denoising

20.5 Numerical Experiments

20.6 Outlook

Appendix

Proof of Theorem 3

References

21 Beta-Boosted Ensemble for Big Credit Scoring Data

21.1 Introduction

21.2 Method Description

21.2.1 Beta Binomial Distribution

21.2.2 Beta-Boosted Ensemble Model

21.2.3 Toy Example

21.2.4 Relation to Existing Solutions

21.3 Experiments

21.4 Conclusion and Future Work

References