Posted:
Do you love Python but hate tracking down bugs in production when time is of the essence? Cloud Debugger can help you identify the root-cause in a few clicks. With our lightning fast, low overhead debugger agent, you simply select the line of code and the debugger returns the local variables and a full stack trace when that line is next executed on any of your instances – all without halting your application or slowing down any requests.

Throughout this year we expanded support for Java projects for Google App Engine and Google Compute Engine. Recently we enabled support for Go projects on Compute Engine. Now Python developers can get in on the fun on App Engine and Compute Engine.

Cloud Debugger adds zero overhead on projects that are not being actively debugged. It also adds less than 10ms to request latency when capturing application state without blocking requests to your application.

With this release, Cloud Debugger is now available for Java, Python and Go projects. Try it out today. Support for additional programming languages and frameworks is in the works.

As always, we’d love direct feedback and will be monitoring Stack Overflow for issues and suggestions.

Posted by Keith Smith, Product Manager

Posted:
For over a decade, we’ve helped evolve the landscape of cloud computing. In that time, we’ve seen plenty of changes — and the past 12 months have been no exception. From widespread adoption of containers to multi-cloud applications, 2015 was truly transformational.

Here, we’ve put together our top moments and themes of the year. Take a look, then tell us on G+ and Twitter what story or trend you’d add, using #CloudTop10.

1. Enterprise, meet Cloud.

For most organizations, Cloud is no longer a question of “if,” but “when”— and according to new estimates, it’ll be sooner than you might think: 34% of enterprises report plans to host over 60% of their apps on a cloud platform in the next two years. In anticipation, most vendors have taken steps to support enterprise workloads. Just look at Microsoft Azure’s partnership with HP and Google’s custom machine types.

2. Containers rush into the mainstream.

Even a year ago, many developers hadn’t yet given containers a try. Fast-forward to 2015 and we saw containers used not just in testing — but widely adopted in production. In fact, container adoption grew 5X in 2015 according to a recent survey. How did this happen so quickly? Part of the answer lies in the availability of robust open-source technologies like Docker and the Kubernetes project. With these technologies, vendors have been able to accelerate container adoption — from VMware’s vSphere integration to Microsoft’s Docker Client for Windows and our own Container Engine.

3. Big Data needs big insights.

In 2015, Big Data didn't live up to the hype. In a May survey, 77% of organizations felt their big data and analytics deployments are failing or have failed to meet expectations. Yet while the finding is clear, the cause is complex. Siloed teams, high maintenance gear and the need for better tools certainly play a part in the problem. What’s the solution? Most likely, it lies in making tooling and data more accessible to citizen data scientists — whose deep domain knowledge can unlock its true value.

4. Machine learning for all.

The potential benefits of machine learning has been evident for a while. Now, thanks to the increased processing power of computers and data centers, that potential is finally being realized. To help spur this evolution on, software libraries (like TensorFlow) are being open-sourced. This’ll allow ideas and insights to be rapidly exchanged through working code, not just research papers.

5. The Future of IoT.

When most of us hear “Internet of Things“ (IoT), we think of the consumer: connecting the thermostat to the watch to the TV and so on. Yet surprisingly, the greatest adoption of IoT is happening in the enterprise. By 2019, it’s estimated that the enterprise market will account for 9.1 billion of 23.3 billion connected devices. That means scale of ingestion and stream-based data processing will become a critical part of every IT strategy—and interest in technologies like Google Cloud Dataflow and Apache Spark is spiking accordingly.

6. API as a business gets big.

Providing application services-on-demand to developers is now a validated business model — as evidenced by the presence of “unicorn” businesses, such as Twilio and Okta. Both companies closed rounds in 2015 at valuations north of $1 billion, and both provide services that developers can incorporate in their applications.

7. Hybrid clouds on the horizon.

Multi-cloud architecture isn’t new: it’s been used for years as a backup and disaster recovery solution. What is new is the rate at which we’re now seeing multi-cloud orchestration tools, like Kubernetes and Netflix’s Spinnaker being widely deployed. This choice helps prevent lock-in to any one vendor — and with estimates that 50% of enterprises will have hybrid clouds by 2017, this trend shows no signs of slowing down.

8. Shifts and shut-downs.

As the Cloud Platform landscape evolves, we’re seeing increasing consolidation in the market. In part, this is likely due to the cloud’s tremendous hardware and engineering demands. Still, one of the biggest announcements of the year came when Rackspace confirmed it will shift focus from their own cloud offering to supporting third-party cloud infrastructures. With the news that HP will officially shut down Helion in January, this is one trend that’s sure to continue through 2016.

9. Going green.

Customers have spoken and they want their cloud green. What’s still up for debate, however, is how to bring the environmental efficiency of larger, pan-regional data centers to local ones — which may not have the scale to be environmentally efficient.

10. What’s yours?

What cloud story or trend would you add to our list? We want to hear from you: submit your idea on G+ and Twitter, using the hashtag #CloudTop10.

We’ll review all the entries, then select a story — and author — to be featured on our blog.

Posted:
The cloud console that our customers use to configure and manage Google Cloud Platform resources provides a single comprehensive location for all GCP services, from App Engine instances to viewing Logs to data processing. But which parts of the platform do you use?

Last month, all GCP customers were invited to start using the new console with features such as pinning and search. With overwhelming positive feedback, we’re pleased to announce its release to general availability.

“Wow! Looks great. I love the way you can pin stuff to the top menu. It makes switching between components much easier (notably App Engine and Datastore). I also like the way you can drill into components, so the UI is less cluttered.” - Gwyn Howell, Appogee HR

“The different areas are very well organized now. Very clean. I love that even the side menu can be searched. That is very useful since there are quite a lot of services.” - Noble Ackerson, LYNXFIT

Thank you for helping us improve the console by providing continual feedback during the beta. After some of you reported page loading latency, we discovered and fixed a bug in Angular Material. We also realigned the color palette to improve the experience after several of you noted that the original red palette for Storage could be misinterpreted as a warning bar.

To quickly review, the updated console now enables you to:

  • Pin each of your commonly-used services to the top of the console for fast access
  • Use the search box and its autocomplete options to easily locate the service you wish to manage
  • Access features in several different ways using the new global navigation options 
    • Open the hamburger menu to see all Cloud Platform offerings in one consolidated place
    • Use the keyboard shortcut (‘/’) to quickly enter into search based navigation
  • Focus solely on a single service and view all content within that service in one place
  • Identify and address issues from a configurable dashboard of your resources and services

When you click in the search bar and start typing, you'll see a dynamically populated set of results:

Figure 1: GCP cloud console search box
Clicking on the menu in the top left will expand the full list of available services and allow you to pin your commonly used items (by clicking on the Pin):
Figure 2: GCP console now allows you to pin favorite services.

We encourage you to give the new console a try and use the feedback button to let us know what you think.

- Posted by Stewart Fife, Product Manager, Google Cloud Platform

Posted:
As part of our constant improvements to the Google Cloud Platform console we’ve recently updated our Google Compute Engine quotas page. Now you can easily see quota consumption levels and sort to find your most-used resources. This gives you a head start on determining and procuring any additional capacity you need so you hit fewer speed bumps on your road to growth and success.
We’ve also improved the process of requesting more quota, which can be initiated directly from the quotas page by clicking on the “Request increase” button. We’ve added additional checks to the request form that help speed up our response processing time; now most requests are completed in minutes. With these changes, we’re making it even easier to do more with Cloud Platform.

You can access your console at https://console.cloud.google.com and learn more about how GCP can help you build better applications faster on the https://cloud.google.com web page.

Posted by Roy Peterkofsky, Product Manager

Posted:
Have you ever wondered how Google Photos helps you find all your favorite dog photos? With today’s release of Google Cloud Vision API, developers can now build powerful applications that can see, and more importantly understand, the content of images. The uses of Cloud Vision API are game changing to developers of all types of applications and we are very excited to see what happens next!

Advances in machine learning, powered by platforms like TensorFlow, have enabled models that can learn and predict the content of an image. Our limited preview of Cloud Vision API encapsulates these sophisticated models as an easy-to-use REST API. Cloud Vision API quickly classifies images into thousands of categories (e.g., "boat", "lion", "Eiffel Tower"), detects faces with associated emotions, and recognizes printed words in many languages. With Cloud Vision API, you can build metadata on your image catalog, moderate offensive content, or enable new marketing scenarios through image sentiment analysis.

The following set of Google Cloud Vision API features can be applied in any combination on an image:

  • Label/Entity Detection picks out the dominant entity (e.g., a car, a cat) within an image, from a broad set of object categories. You can use the API to easily build metadata on your image catalog, enabling new scenarios like image based searches or recommendations.
  • Optical Character Recognition to retrieve text from an image. Cloud Vision API provides automatic language identification, and supports a wide variety of languages.
  • Safe Search Detection to detect inappropriate content within your image. Powered by Google SafeSearch, the feature enables you to easily moderate crowd-sourced content.
  • Facial Detection can detect when a face appears in photos, along with associated facial features such as eye, nose and mouth placement, and likelihood of over 8 attributes like joy and sorrow. We don't support facial recognition and we don’t store facial detection information on any Google server.
  • Landmark Detection to identify popular natural and manmade structures, along with the associated latitude and longitude of the landmark.
  • Logo Detection to identify product logos within an image. Cloud Vision API returns the identified product brand logo, with the associated bounding polybox.


You can currently call the API by embedding an image as part of the request. In future phases, we will add support for integrating with Google Cloud Storage. The Vision API enables you to request one or more annotation types per image.
To show a simple example of the Vision API, we have built a fun Raspberry Pi based platform with just a few hundreds of lines of Python code, calling the Vision API. Our demo robot can roam and identify objects, including smiling faces. This is just one simple example of what can be done with Cloud Vision API:


Aerosense, a subsidiary of Sony Mobile Communications Inc, was among the first early testers to use Cloud Vision API and had some initial feedback to share:


To join the Limited Preview, please sign up here. We cannot wait to see what amazing applications you build with Vision API, and we look forward to hearing from you!

- Posted by Ram Ramanathan, Product Manager, Google Cloud Platform

Posted:
Containers are all the rage right now. There are scores of best practices papers and tutorials out there, and "Intro to Containers" sessions at just about every conference even tangentially related to cloud computing. You may have read through the Docker docs, launched an NGINX Docker container, and read through Miles Ward’s Introduction to containers and Kubernetes piece. Still, containers can be a hard concept to internalize, especially if you have an existing application that you’re considering containerizing.

To help you through this conceptual hurdle, I’ve written a four-part series of blog posts that gives you a hands-on introduction to building, updating, and using containers for something familiar: running a Minecraft server. You can check them out here:


In the first part of the series, you’ll learn how to create a container image that includes everything a Minecraft server needs, use that image on Google Compute Engine to run the server, and make it accessible from your Minecraft client. You’ll use the Docker command-line tools to build, test, and run the container, as well as to push the image up into the Google Container Registry for use with a container-optimized instance.


Next, you'll work through the steps needed to separate out storage from the container and learn how to make regular backups of your game. If you’ve ever made a mistake in Minecraft, you know how critical being able to restore world state can be! As Minecraft is always more fun when it’s customized, you'll also learn how to update the container image with modifications you make to the server.properties file.

Finally, you’ll take the skills that you’ve learned and apply them to making something fun and slightly absurd: Minecraft Roulette. This application allows you to randomly connect to one of several different Minecraft worlds using a single IP as your entry point. As you work through this tutorial, you’ll learn the basics of Kubernetes, an open source container orchestrator.

By the end of the series, you’ll have grasped the basics of containers and Kubernetes, and will be set to go out and containerize your own application. Plus, you’ll have had the excuse to play a little Minecraft. Enjoy!

This blog post is not approved by or associated with Mojang or Minecraft.

Posted by Julia Ferraioli, Senior Developer Advocate, Google Cloud Platform

Posted:
Logs generated by applications and services can provide an immense amount of information about how your deployment is running and the experiences your users are having as they interact with the products and services. But as deployments grow more complex, gleaning insights from this data becomes more challenging. Logs come from an increasing number of sources, so they can be hard to collate and query for useful information. And building, operating and maintaining your own infrastructure to analyze log data at scale requires extensive expertise in running distributed systems and storage. Today, we’re introducing a new solution paper and reference implementation that will show how you can process logs from multiple sources and extract meaningful information by using Google Cloud Platform and Google Cloud Dataflow.

Log processing typically involves some combination of the following activities:

  • Configuring applications and services
  • Collecting and capturing log files
  • Storing and managing log data
  • Processing and extracting data
  • Persisting insights

Each of those components has it’s own scaling and management challenges, often using different approaches at different times. These sorts of challenges can slow down the generation of meaningful, actionable information from your log data.

Cloud Platform provides a number of services that can help you to address these challenges. You can use Cloud Logging to collect logs from applications and services, and then store them in Google Cloud Storage buckets or stream them to Pub/Sub topics. Dataflow can read from Cloud Storage or Pub/Sub (and many more), process log data, extract and transform metadata and compute aggregations. You can persist the output from Dataflow in BigQuery, where it can be analyzed or reviewed anytime. These mechanisms are offered as managed services—meaning they can scale when needed. That also means that you don't need to worry about provisioning resources up front.

The solution paper and reference implementation describe how you can use Dataflow to process log data from multiple sources and persist findings directly in BigQuery. You’ll learn how to configure Cloud Logging to collect logs from applications running in Container Engine, how to export those logs to Cloud Storage, and how to execute the Dataflow processing job. In addition, the solution shows you how to reconfigure Cloud Logging to use Pub/Sub to stream data directly to Dataflow, so you can process logs in real-time.


Check out the Processing Logs at Scale using Cloud Dataflow solution to learn how to combine logging, storage, processing and persistence into a scalable log processing approach. Then take a look at the reference implementation tutorial on Github to deploy a complete end-to-end working example. Feedback is welcome and appreciated; comment here, submit a pull request, create an issue, or find me on Twitter @crcsmnky and let me know how I can help.

- Posted by Sandeep Parikh, Google Solutions Architect