Is Your Favorite Open Source Technology in Growth or Decline?

by Kaitlin Pike

on

This week we’re at OSCON in Austin as part of our efforts to support the open source community. While we’re here, we thought we’d compare what’s trending on social media versus what we can see through our new Stack Overflow Trends tool. We launched Trends this week so developers could see how technologies have grown or shrunk over time.

The top terms on social yesterday included a combination of companies and tech, with Kubernetes and TensorFlow leading the pack for the latter group. Here’s a short list of some of the top terms:

 

Early in the day, we presented how Docker and Kubernetes were tracking since we’ve seen buzz about both at OSCON.

But more impressive is the explosion in popularity of TypeScript, a superset of JavaScript that we use internally at Stack Overflow.

Another interesting trend we noticed yesterday was the expansion of several data science technologies, especially the fast-growing open-source TensorFlow library, which recently caught up with the MATLAB language in questions asked per month.

Yesterday at Microsoft’s Build conference (it’s a busy week for tech events), the company announced an Azure database for MySQL and PostgresSQL. Unsurprisingly, MySQL is still popular, although its growth may have stalled or even started shrinking.

With all the work we’ve done to launch Stack Overflow Trends, we were pretty pleased that O’Reilly Media (producers of OSCON) founder and CEO Tim O’Reilly gave us a shout out. (Thanks, Tim.)

He later asked us how competing cloud technologies have been growing on the platform. Short story:

Tim also pointed out that we should include some context for the analysis and how the Stack Overflow community might differ from the overall programmer population.

That’s an important question that we often think about and will share more about in the future. Though in regards to this specific example, since Stack Overflow runs on .NET and its founders were well known for their Microsoft-related blogging, the community was originally shifted towards Microsoft developers. But that appears to have change over time, with Microsoft technologies shrinking as a share of questions.

 

[Update to the above]

We’ll be back with more analysis on Twitter today and tomorrow.

Don’t see your favorite language, technology, or framework in this post? Use the Stack Overflow Trends tool to create your own graphs, and see what you can learn about how the developer ecosystem is changing and where it might be going in the future.

By Kaitlin Pike, Developer Marketing Manager

Tagged in  Announcements Company Data Engineering

  • svick

    The last link has a typo: it should be “windows”, not “window”.

    • Kaitlin

      Fixed!

      • Manish Kumar Rathore

        Readlly Very Creative blog I have ever seen on Internet. Can you please tell how can we use two whats app accounts in same mobile. And my second Question is How can I give border around every blogpost for my blog. my blog is http://www.sadupayog.com. Please reply.

  • Oli Barnett

    Disclaimer: I’m not much of a dab hand with statistics, and I would be happy to have this explained to me by a grownup.

    I can see how the ‘growth’ part of these trends makes sense, as an increase in questions logically follows an uptick in adoption – if that technology is not yet covered in depth on SO.

    That ‘if’ is what makes me wonder whether it necessarily follows that a decrease in questions means a technology is ‘declining’, however? The longer the technology has been on SO, the more potential questions will already have been asked and answered. Then users who have a question will be able to get an answer without actually asking it themselves. So an eventual decline in questions per year seems (to me) to be the natural course of events – even if the technology in question is still being adopted at an increasing rate.

    Obviously there are other factors, such as new features, updates etc. However, once a technology is relatively established and mature, and once all of the ‘big’ questions have already been answered, I would be surprised if new questions would come in fast enough to maintain the rate of the initial surge.

    • Alex Thurston

      Yeah you’re right about the points you make and those should be considered. However, I think that they are somewhat implied in the article; the article isn’t trying to state that this is more than just statistics being analyzed broadly for correlations. Overall, the data doesn’t seem that compelling to me for the reasons you mentioned, but it is interesting to think about for what it is, and more analysis and data will probably help us learn more about how to understand the popularity of frameworks in the future.

    • Arzael

      I think we can consider MySQL as a pretty mature technology, as it is there for more than 20 years, now.

      Anyway, since 2010, its question flow does not fluctuate that much.

      If such a mature tech is able to keep getting a constant flow of question, it seems possible to say that the “question pool” for a technology is so big that we can approximate it to be infinite without getting too big of an error. (I mean… For windows techs, losing almost 8-9% is BIG o.O)

      Or maybe it is because of other factors (Being taught everywhere, and having some misconception tagging between SQL and MySQL, for exemple)

    • David Robinson

      I’ve often heard this hypothesis (maybe most of the questions have been answered). I think it’s part of the story for some technologies like C#, which started out with a very large share of the site’s community. But there are two reasons I’m inclined to think these rises and declines are meaningful:

      1. The two most asked about tags on the site are Java and Javascript: Java’s been pretty steady for the site’s history, and Javascript has been growing (though it may have leveled off this year). If it really were possible to “ask all the relevant questions” about a technology, surely we would have hit that sometime in the 1.3 million Javascript questions!

      2. We don’t make question traffic data public (it’s relevant to our consulting product Insights https://insights.stackoverflow.com/ ) but we can look at the questions visited in the last five years by language, and the trends generally match (though they sometimes lag). As people stop asking about a technology, you also see fewer visits to existing ones. (If you’re interested in seeing it for a few tags I could certainly make a graph for you!)

      Bjarne Stroustrup once said “There are only two kinds of languages: the ones people complain about and the ones nobody uses.” I’d say the same typically applies to languages asked about on Stack Overflow.

      • Oli Barnett

        Thanks David! Interesting to know that traffic correlates after all, I was wondering about that.

  • Cypress Frankenfeld

    Could we see similar graphs, but instead of questions asked, could we see visits and activity on new and existing questions? I feel that would better represent the ‘decline’ of technologies.

  • Edward Brey

    The moral of the story is that if you want you developer technology to appear wildly successful, be sure to skimp on the documentation. 🙂

  • Naldru

    I don’t think that the graphs can be easily used to determine whether technologies are growing or declining in use. Skimping on the documentation will cause people to use Stack Overflow, especially if it is combined with unstable interfaces having features added or removed (or at least deprecated) every few years. The scary thing is that the idea of maintaining documentation appears to suffer from the idea that “it isn’t a bug if you can convince people that it’s a feature”. “Convention over configuration” would be much more impressive if somebody could actually explain what the conventions are. When I see programming languages that don’t have formal syntaxes, I don’t see how anyone can properly document how they work. The main problem with SOAP was that people used standard tools to create the schemas and left them incomplete or ambiguous. If your WSDL or DTD or other document contains tempuri.com, it is wrong and has a good chance of failing when used in conjunction with other technologies. (Look up the domain on wikipedia or StackOverflow if you don’t believe me.) Encouraging people to use technologies without understanding them will cause a lot of referrals to Stack Overflow.

    I also think that you should be looking at percentage of views as well as percentage of questions. A very high number of views for a small number of questions may indicate a subject that is inadequately covered in a confusing way. Look at Goodhart’s Law. https://en.wikipedia.org/wiki/Goodhart%27s_law People will use these charts to show that their preferred technology is the right choice.

    By the way, if you really wanted to get some useful information, you could give a survey to people selected randomly who viewed a specific topic. The old “why did you come to this site”, “why did you view this page”, and “did you solve your problem”. You could also add “what other techniques did you use to solve your problem” and “why didn’t they solve your problem”.

    In terms of the scientific method, charts like this might be useful in developing hypothesis and designing experiments, but are limited in the type of information that they can yield without further analysis.

  • Bella

    Minnesota Vikings

    Saints RB Adrian Peterson aims for big game against Vikings in Week 1

    During his 10 years with the Minnesota Vikings, Adrian Peterson often ribbed the team’s defenders, telling them about the big numbers he would post against them if he ever got the chance.

    Now that Peterson is with the New Orleans Saints, he’ll finally have the opportunity to back up his boasts.

    In an interview with the Saints’ website on Wednesday, Peterson joked his return trip to U.S. Bank Stadium on Sept. 11 will be “revenge,” then said he’s looking forward to showing the Vikings’ defensive players what he can do against them.

    I’ll hit you guys for 150 or 200 yards,”It’s going to be fun to go against those guys so I can let them know [when] I was doing that talking, telling them,’ that I wasn’t just talking, ‘Hey,Robert Nkemdiche Jersry ,” he said. “They’ll see.”

    when New Orleans opens the season in Minnesota on Monday Night Football. The schedule was released five days before the running back signed with the Saints,Peterson,D.J. Humphries Jersey, who signed a two-year deal with the Saints on April 25,
    Markus Golden Jersey     , but as Peterson went through his final negotiations with New Orleans, will play his first regular-season game for a team other than the Vikings on Sept. 11, he took the Week 1 trip to Minnesota as a sign of kismet.

    “It’s going to be exciting, this is meant to be.’ So many great memories down there in Minnesota. Great fan base. They supported me through everything, it’s just ironic. When the schedule came out and I saw that,” he said. “You know, I was like, throughout my career, ‘Wow, so just to embrace the love once I get there will be exciting.”

    The 32-year-old running back figures to split carries with Mark Ingram and Alvin Kamara,Authentic Robert Nkemdiche Jersey,Authentic Brandon Williams Jersey, and he typically hasn’t been the fastest starter in the league; he hasn’t rushed for more than 90 yards in Week 1 since 2013 and hasn’t surpassed 100 yards in a regular-season opener since 2009. When Peterson returned from his 2014 suspension in Week 1 of the 2015 season on Monday Night Football,Authentic D.J. Humphries Jersey, he admitted to being too excited and ran for just 31 yards on 10 carries as the Vikings rolled out their ill-fated experiment with the running back in the shotgun.

    he could get some Week 1 opportunities against a Vikings run defense that has been the weak link of an otherwise stout unit in recent years; the Vikings were 20th in the league against the run last year,Authentic MarkusGolden Jersey,Authentic Keanu Neal Jersey,If he can keep his excitement in check come September,Keanu Neal Jersey, however, and they haven’t ranked in the top half of the league in the category during any of Mike Zimmer’s three years as head coach.

  • Lundin

    On “petoverflow.com”, there were each year 100 questions about dogs and 100 questions about cats, each taking up 50% of the site’s questions. Due to a recent hype, pigs became popular in the last year and no less than 400 pig questions were posted. While at the same time the questions about dogs and cats also increased to 200 each. By “scientific proof through irrelevant graph” we can therefore show that the popularity of dogs and cats have dropped by half!

  • Fawad Raza

    TypeScript’s “explosion” is artificial to be honest. Had Angular not made it theirs it’d be at par with likes of C# etc. in my view.

    • Laurent

      You can say about any programming language though, no? They’re all popular for a reason, which sometime is quite random.

      • Florian Reuschel

        I’m looking at you, Ruby.

        • Lopicycle

          *cough* PHP *cough*

    • abram_k

      Another way of looking at it – perhaps Typescript was so good that Angular adopted it.

      • peterh

        In my opinion, both Typescript and Angular are catastrophic – the only reason to use them, that some yet bigger trash would take their position if they would disappear.

        In my opinion, a community-based fork of the gwt had solved everything what is needed.

        • Vassilis

          “Catastrophic” is a big statement…
          Can you name one “catastrophic” event that those two are responsible? Event of any kind…

          I think you are overreacting without a reason.
          Both are solving issues that are not only related with the code part but also architectural decisions, design problems etcetera that will eventually produce better software and better experiences for developers & users.

          Both are also open-source and both if you use them properly are great tools to work with.

          • peterh

            Yes. For example, if next morning our boss will come and see, how our project works. And instead of doing our task, we have to fight with their change detector. Another real life example: we switch from angular2 to angular4. They say it is compatible, so we upgrade. Preparing the worst case, we plan a day for the switch. And it takes a week of fight, meanwhile I have to make a local fork of the zonejs and cherrypick an unreleased commit into it. And a lot of fight, again, with the change detector. Another real life example, my boss asks, why the hell is our project 4.5 megabyte compiled, while our source is only some hundred K. Note, we use the best configured latest stable rollup. I invest another day to make it compiled by google closure. After a wasted week, which resulted a not working, but only 1 megabyte big final js, I suspend the subproject. These all were catastrophal to me.

          • Vassilis

            We still your reply seems that you had some bad architectural decisions.
            Angular build is quite small but if for example you use a ton of libraries just to use a small function of each instead of writing yourself then yes you can get a big build.
            Also one week for update to 4 from 2 also seem that either you haven’t read the changelog or that your project had issues. A one day update is more than possible even in huge projects.

            I’d love to read a blog of yours to read the actual technical challenges that you had and discuss on that base instead of speaking in general.

          • peterh

            “Angular build is quite small” <- it seems you never tried Angular, I think it is the time to begin. Before it, look around on the Internet, what others say. What I say, is not archivable.

          • Vassilis

            Ye one more correct statement. It seems that you stayed in the alpha versions. As for me ye you got to the point I never used it… Only 10+ projects and some contribution on the repo. I’ll try to start.

            First read the try and the try to insult again you noob.

          • peterh

            But I could also mention, as they silently forbade the support of self-closed tags in templates, with a cryptic error message, “to follow html5 more closely”. Or that they changed API names and partially architecture around 2.0-rc3. Note, in RC!!! The angular is the oracle jsf generator, in a new mantle, so is it. Or, that you simply can’t circularly import @Component s. The only reason to use it, that it pays, but the sad truth is that we could work more fast with pure js, or with jquery.

            If you use a framework, you have to ask: in which ratio of your work time is the framework helping you, and in which ratio you have to fight it. Working with angular actually means that you have to fight it. It is one of the worst frameworks I’ve ever faced.

          • peterh

            Note also that you simply can’t index in a map anything with anything. Or, that you practically can’t use enums.Well, these are typescript trashes.

    • David Conrad

      “If people weren’t using it, it wouldn’t be as popular.”

      • Fawad Raza

        Sorry but one of replies here looked at what I wrote, yours, worse so in my view. What I said, was simply pointing out that a thing like Angular did not need a crutch to be popular. Typescript’s “uses” statistic is forgetting that many are using it *just* because it is included in Angular and vice versa isn’t the case.

  • David Forck

    a part of me wonders if questions by month is really that great a metric for popularity of a language. for example, c# has been around on SO since before 2009, so it has a sizeable amount of questions on the site. if the questions asked per month could somehow be combined with number of web searches/references/# of views for questions with that tag, I think we’d get a lot better insight into the popularity of these languages.

  • Ajay Singh

    Wouldn’t bat a rat if you were saying “Since we began using it internally, it now shoots at 0.80 percent”. Hmm. Big numbers, too big to fail!

  • Carlos Araya

    How many of those technologies are down because people are afraid to ask questions less the incur the wrath of the experts who berate people for not asking the question correctly (forgetting that most newbies don’t really know the technology work and may not really know what items are important when asking a question)?

    The reverse is also true, how many of those technologies go up because development teams use SO as their forum for technical support? How many of those forums work and how many don’t?

    • Laurent

      There’s lot of terrible questions on Stack Overflow every day (for example, in the PHP category I’d say more than 50% are in this category), so it doesn’t look like anybody is afraid of asking questions.

  • Thanks for sharing!

  • Nick Kartha

    Mozilla Open Badges are in decline. Please help us!