Answering your question with a question: what exactly is machine learning? Trevor Hastie, Robert Tibshirani and Jerome Friedman in The Elements of Statistical Learning, Kevin P. Murphy in Machine Learning A Probabilistic Perspective, Christopher Bishop in Pattern Recognition and Machine Learning and a number of other machine learning "bibles" mention linear regression as one of the machine learning "algorithms". Machine learning is partly a buzzword for applied statistics and the distinction between statistics and machine learning is often blurry.

Linear regression is definitely an algorithm that can be used in machine learning. But, reductio ad absurdum: Anyone with a copy of Excel can fit a linear model.

Even restricting ourselves to linear models, there are a few more things to consider when discussing machine learning:

• Machine learning on business problems may involve a lot more data. "Big data", if you want to use the buzzword. Cleaning and preparing the data may take more work than the actual modelling. And when the volume of data exceeds the capacity of a single machine to process it then the engineering challenges are as significant as the statistical challenges. (Rule of thumb: if it fits in main memory it's not big data).
• Machine learning often involves many more explanatory variables (features) than traditional statistical models. Perhaps dozens, sometimes even hundreds of them, some of which will be categorical variables with many levels. When these features can potentially interact (e.g. in a cross effects model) the number of potential models to be fit grows rapidly.
• The machine learning practitioner is usually less concerned with the significance of individual features, and more concerned with squeezing as much predictive power as possible out of a model, using whichever combination of features does that. (P-values are associated with explanation, not prediction.)
• With a large number of features, and various ways of engineering those features, model selection by hand becomes infeasible. In my opinion, the real challenge in machine learning is the automated selection of features (feature engineering) and other aspects of model specification. With a linear model there are various ways of doing this, usually variants of brute force; including step-wise regression, back elimination etc, all of which again require significant computing power. (Second rule of thumb: if you are selecting features by hand, you are doing statistics, not machine learning).
• When you automatically fit many models with many features, over-fitting is a serious potential issue. Dealing with this problem often involves some form of cross validation: i.e. yet more brute force computation!

The short answer, from my point of view, is that where machine learning deviates from traditional statistical modelling is in the application of brute force and numerical approaches to model selection, especially in domains with a large amount of data and a large number of explanatory variables, with a focus on predictive power, followed by more brute force for model validation.

I think Mitchell's definition provides a helpful way to ground the discussion of machine learning, a sort of first principle. As reproduced on Wikipedia:

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

This is helpful in a few ways. First, to your immediate question: Regression is machine learning when its task is to provide an estimated value from predictive features in some application. Its performance should improve, as measured by mean squared (or absolute, etc.) held out error, as it experiences more data.

Second, it helps delineate machine learning from related terms, and its use as a marketing buzzword. Contrast the task above with a standard, inferential regression, wherein an analyst interprets coefficients for significant relationships. Here the program returns a summary: coefficients, p-values, etc. The program cannot be said to improve this performance with experience; the task is elaborate calculation.

Finally, it helps unify machine learning sub fields, both those commonly used in introductory exposition (supervised, unsupervised) with others like reinforcement learning or density estimation. (Each has a task, performance measure and concept of experience, if you think on them enough.) It provides, I think, a richer definition that helps delineate the two fields without unnecessarily reducing either. As an example, "ML is for prediction, statistics for inference" ignores both machine learning techniques outside supervised learning, and statistical techniques that focus on prediction.

Common view is that machine learning made up of 4 areas:

1) Dimensionality Reduction

2) Clustering

3) Classification

4) Regression

Linear regression is a regression. Once the model is trained it could be used for predictions, like any other, say, Random Forest Regression.

There's no law that says that a cabinet maker can't use a barrel maker's saw.

Machine learning and statistics are vague labels, and if well-defined there is a lot of overlap between statistics and machine learning. And this goes for methods of these two areas as well as (and separately) for people who label themselves with these two areas.

Linear regression is a very well defined mathematical procedure. I tend to associate it with the area of statistics and people who call themselves 'statisticians' and come out of academic programs with labels like 'statistics'. SVM (Support Vector Machines) is likewise a very well defined mathematical procedure that has some every similar inputs and outputs and solves similar problems. I tend to associate it however with the area of machine learning and people who call themselves computer scientists (or people who work in artificial intelligence or machine learning which tend to be considered part of computer science as a discipline).

But some statisticians might use SVM, and some AI people use logistic regression. (and just to be clear, it is more likely that a statistician or AI researcher would develop a method than actually put it to practical use).

Frankly, all the methods of machine learning I would put squarely inside the domain of statistics. Even such recent things like Deep Learning, RNNs, CNNs, LSTMs, CRFs, etc. Those are al predictive modelling methods usually labeled with 'machine learning; and rarely associated with statistics. But they are predictive models, with the allowance that they can be judged using statistical methods.

But, yes, I see and sometimes share your distaste for the misapplication of these words. Linear regression is such a fundamental part of things called statistics that it feels very strange and misleading to call its use 'machine learning'.

To illustrate, Logistic regression is identical mathematically to a Deep Learning network with no hidden nodes and the logistic function as the activation function for the single output node. I wouldn't call logistic regression a machine learning method, but it is certainly used in machine learning contexts.

It's like saying, when washing a window with water that you're using quantum chemistry. Well yeah sure that's not technically wrong but you're implying a lot more than what's needed.

I'll argue that the distinction between machine learning and statistical inference is clear. In short, machine learning = prediction of future observations; statistics = explanation.

Here is an example from my field of interest (medicine): when developing a drug, we search for gene(s) which best explain a disease state, with the goal of targeting it/them with the drug. We use statistis for that. In contrast, when developing diagnostic tests, for example predicting whether the drug will help a patient, the goal is strictly finding the best predictor of the future outcome, even if it comprises many genes and is too complicated to understand. We use machine learning for this purpose. There are multiple published examples [1], [2], [3], [4] showing that presence of the drug target is not a good predictor of the treatment outcome, hence the distinction.

Based on this, it is fair to say that one is doing machine learning when the goal is strictly predicting outcome of future/previously unseen observations. If the goal is understanding a particular phenomenon, then that is statistical inference, not machine learning. As others have pointed out, this is true regardless of the method involved.

To answer your question: in the specific research that you describe, the scientists were comparing the factor roles (weights) in different linear regression models, not comparing the model accuracies. Therefore, it is not accurate to call their inference machine learning.

[1] Messersmith WA, Ahnen DJ. Targeting EGFR in Colorectal Cancer. The New England Journal of Medicine; 2008; 359; 17.

[2] Pogue-Geile KL et al. Predicting Degree of Benefit From Adjuvant Trastuzumab in NSABP Trial B-31. J Natl Cancer Inst; 2013; 105:1782-1788.

[3] Pazdur R. FDA Approval for Vemurafenib. https://www.cancer.gov/about-cancer/treatment/drugs/fda-vemurafenib. Updated July 3, 2013.

[4] Ray T. Two ASCO Studies Show Challenge of Using MET Signaling as Predictive Marker in NSCLC Drug Trials. GenomeWeb, June 11, 2014.

Linear regression is a technique, while machine learning is a goal that can be achieved through different means and techniques.

So regression performance is measured by how close it fits an expected line/curve, while machine learning is measured by how good it can solve a certain problem, with whatever means necessary.

It can be useful to call linear regression machine learning because doing so generally implies a couple important things about how you went about solving your problem:

1. You decided it wasn't necessary to check causal assumptions and prior theory behind your explanatory variables. It signals that your model was not intended to explain but to predict. This is perfectly reasonable in a lot of settings, for example, predicting email spam based on keywords. There isn't really a lot of literature on which words predict spam, and there are so many words it doesn't make sense to think through the theoretical significance of each word
2. You didn't check for variable significance or use p-values but instead likely opted for a holdout set or cross validation to assess out-of-sample predictive performance. This can be perfectly valid if - back to the email spam example - if really all you care about is producing a model that effectively predicts spam, even if this comes at at the cost of including variables that might not pass traditional significance tests.

However, if your model is more intended to explain than predict, and you do rigorously check your model's theoretical causal assumptions, etc then yes, it is rather silly to call it machine learning.

It's the Machine, Stupid!

I am neither a statistician nor a Big Data(TM) expert. However, I would say that the essential distinction is that "machine learning" requires "a machine". In particular, it implies agency. The result will not be consumed leisurely by a human. Rather, the result will be the input to a closed cycle whereby an automated system improves its performance.

Closed System

This is very much in line with Sean Easter's answer, but I just want to emphasize that in commercial applications, a machine is looking at the results and acting on them. A classic example is the CineMatch algorithm which was the target of the Netflix Prize. A human could look at the output of CineMatch and learn interesting features about movie viewers. But that is not why it exists. The purpose of CineMatch is to provide a mechanism whereby Netflix servers can suggest movies to customers that they will enjoy. The output of the statistical model goes into the recommender service, which ultimately produces more input as customers rate movies, some of which were selected on the advice of CineMatch.

Open System

On the other hand, if a researcher uses an algorithm to produce statistical results which are displayed in a presentation to other humans, then that researcher is most decidedly not engaging in machine learning. This is, quite obviously to me, human learning. The analysis is performed by a machine, but it is not a machine that is doing the learning, per se. Now, it is "machine learning" to the extent that a human brain did not experience all of the sample inputs and derive the statistical results "biologically". But I would call it "statistics" because this is exactly what statisticians have been doing since the field was invented.

Conclusion

Thus, I would answer this question by asking: "Who consumes the results?" If the answer is: "humans", then it's "statistics". If the answer is: "software", then it's "machine learning." And when we say that "software consumes the results", we don't mean that it stores it somewhere for later retrieval. We mean that it performs behavior which is determined by the results in a closed loop.