Blog posts tagged 'Data'
Using Apache Spark for large-scale language model training
The pipeline is modular, readable, and more maintainable, with reductions in both resource usage and data landing time. Read more...
Beringei: A high-performance time series storage engine
Beringei powers most of the performance and health monitoring at Facebook while enabling engineers and analysts to make decisions quickly with accurate, real-time data. Read more...
Open population datasets and open challenges
Sharing data about how people are aggregated around the world can help solve challenges such as connectivity, infrastructure planning, humanitarian aid, and disaster response. Read more...
A comparison of state-of-the-art graph processing systems
The study measured the relative performance and ability of two systems to handle large graphs, focusing on performance and usability. Read more...
Highlights from @Scale 2016
Engineers representing hundreds of companies gathered to discuss the challenges and opportunities of building apps and systems at scale. Read more...
Facebook announces new tech at @Scale 2016
New data storage technologies, 360 video improvements, and performance tools were revealed throughout the day. Read more...
Apache Spark @Scale: A 60 TB+ production use case
Through a series of performance and reliability improvements, we were able to scale Spark to handle a TB-scale entity ranking system in production. Read more...
MyRocks: A space- and write-optimized MySQL database
Deploying MyRocks to a database tier in one of our data center regions enabled a 50 percent reduction in storage requirements. Read more...
Smaller and faster data compression with Zstandard
With a performance-first design optimized for modern CPUs, Facebook's new compression algorithm translates directly to faster data transfer and smaller storage requirements. Read more...
@Scale 2016 lineup announced!
Registration for the 2016 @Scale conference is now open. Read more...
Lighting the way to deep machine learning
Open source Torchnet helps researchers and developers build rapid and reusable prototypes of learning systems in Torch. Read more...
Data @Scale, June 2016 — Recap
Engineers working on large-scale storage systems and analytics discuss challenges and collaborate on new solutions. Read more...
Introducing DeepText: Facebook's text understanding engine
DeepText can understand with near-human accuracy the textual content of several thousand posts per second, spanning more than 20 languages. Read more...
Facebook Chef cookbooks
This suite of cookbooks — along with a sample 'init' cookbook — will allow anyone who wants to use our model of Chef in their own environment to get started easily and quickly. Read more...
Facebook Boston tech talk: GraphQL
How we built Facebook Lite for every Android phone and network
FB Lite is the fastest-growing version of Facebook, 100 million users in under nine months. Read more...
Connecting the world with better maps
By applying computer vision techniques to satellite imagery, we can identify how populations are distributed in remote locations and determine the best way to provide connectivity in those areas. Read more...
NetNORAD: Troubleshooting networks via end-to-end probing
NetNORAD troubleshoots issues independently of device polling to help keep Facebook's massive networking infrastructure up and running. Read more...
Favorite hacks of 2015
The passion people have for ideas generated at hackathons results in everything from new products to open source tools. Read more...
Improving software RAID with a write-ahead log
Software RAID has some drawbacks, which can be problematic at Facebook's scale. Using a write-ahead log can address some of these issues and improve reliability of the array. Read more...
Under the hood: Broadcasting live video to millions
Solving for traffic spikes through load balancing and enabling RTMP playback to bring latency down to a few seconds are some of the ways we enabled seamless live video sharing on Facebook. Read more...
Building the Presto community
When we launched Presto, we saw dramatic query performance improvement across multiple internal Hadoop clusters. Read more...
Instrumenting meetings at Facebook
Data from calendars, motion sensors, and videoconferencing allows us to analyze how meeting space is utilized so employees get the resources they need. Read more...
Inside @Scale 2015
A thousand engineers from hundreds of companies joining together to share lessons learned and best practices for building systems and applications at scale. Read more...
GraphQL: A data query language
GraphQL is a data query language and runtime designed and used at Facebook to request and deliver data to mobile and web apps since 2012. Read more...
Relay: Declarative data for React applications
We've been working on a solution to simplify the process of retrieving server data. Read more...
The Parse SDK: What's inside?
In this post, we'll unpack a few of the most challenging aspects of building the Parse SDKs — structuring an asynchronous API, decoupling architecture, and achieving API consistency. Read more...