- The compositor didn’t have a timely way of knowing when a video frame needed display. Video frames were selected on the media pipeline thread while the compositor would occasionally come along looking for them on the compositor thread, but if the compositor thread was busy it wouldn’t get the notification on time.
- Chrome’s media pipeline didn’t know when the compositor would be ready to draw its next new frame. This led to the media pipeline sometimes picking a frame that was too old by the time the compositor displayed it.
Video thumbnails are often the first things viewers see when they look for something interesting to watch. A strong, vibrant, and relevant thumbnail draws attention, giving viewers a quick preview of the content of the video, and helps them to find content more easily. Better thumbnails lead to more clicks and views for video creators.
Inspired by the recent remarkable advances of deep neural networks (DNNs) in computer vision, such as image and video classification, our team has recently launched an improved automatic YouTube "thumbnailer" in order to help creators showcase their video content. Here is how it works.
The Thumbnailer Pipeline
While a video is being uploaded to YouTube, we first sample frames from the video at one frame per second. Each sampled frame is evaluated by a quality model and assigned a single quality score. The frames with the highest scores are selected, enhanced and rendered as thumbnails with different sizes and aspect ratios. Among all the components, the quality model is the most critical and turned out to be the most challenging to develop. In the latest version of the thumbnailer algorithm, we used a DNN for the quality model. So, what is the quality model measuring, and how is the score calculated?
|The main processing pipeline of the thumbnailer.|
(Training) The Quality Model
Unlike the task of identifying if a video contains your favorite animal, judging the visual quality of a video frame can be very subjective - people often have very different opinions and preferences when selecting frames as video thumbnails. One of the main challenges we faced was how to collect a large set of well-annotated training examples to feed into our neural network. Fortunately, on YouTube, in addition to having algorithmically generated thumbnails, many YouTube videos also come with carefully designed custom thumbnails uploaded by creators. Those thumbnails are typically well framed, in-focus, and center on a specific subject (e.g. the main character in the video). We consider these custom thumbnails from popular videos as positive (high-quality) examples, and randomly selected video frames as negative (low-quality) examples. Some examples of the training images are shown below.
|Example training images.|
Compared to the previous automatically generated thumbnails, the DNN-powered model is able to select frames with much better quality. In a human evaluation, the thumbnails produced by our new models are preferred to those from the previous thumbnailer in more than 65% of side-by-side ratings. Here are some examples of how the new quality model performs on YouTube videos:
|Example frames with low and high quality score from the DNN quality model, from video “Grand Canyon Rock Squirrel”.|
|Thumbnails generated by old vs. new thumbnailer algorithm.|
We recently launched this new thumbnailer across YouTube, which means creators can start to choose from higher quality thumbnails generated by our new thumbnailer. Next time you see an awesome YouTube thumbnail, don’t hesitate to give it a thumbs up. ;)
Weilong Yang, software engineer, recently watched “Contact Juggling - His Skills are Totally Hypnotizing”
Min-hsuan Tsai, software engineer, recently watched ”People Are Awesome 2015”
Thanks to the Video Content Analysis and YouTube Creator teams
How to start developing
- Choose your reports:
- Video reports provide statistics for all user activity related to a channel's videos or a content owner's videos. For example, these metrics include the number of views or ratings that videos received. Some video reports for content owners also include earnings and ad performance metrics.
- Playlist reports provide statistics that are specifically related to video views that occur in the context of a playlist.
- Ad performance reports provide impression-based metrics for ads that ran during video playbacks. These metrics account for each ad impression, and each video playback can yield multiple impressions.
- Estimated earnings reports provide the total earnings for videos from Google-sold advertising sources as well as from non-advertising sources. These reports also contain some ad performance metrics.
- Asset reports provide user activity metrics related to videos that are linked to a content owners' assets. For its data to included in the report, a video must have been uploaded by the content owner and then claimed as a match of an asset in the YouTube Content ID system.
- Schedule reports:
- Get an OAuth token (authentication credentials)
- Call the reportTypes.list method to retrieve a list of the available report types
- Create a new reporting job by calling jobs.create and passing the desired report type (and/or query in the future)
- Retrieve reports:
- Get an OAuth token (authentication credentials)
- Call the jobs.list method to retrieve a list of the available reporting jobs and remember its ID.
- Call the reports.list method with the jobId filter parameter set to the ID found in the previous step to retrieve a list of downloadable reports that that particular job created.
- Creators can check the report’s last modified date to determine whether the report has been updated since the last time it was retrieved.
- Fetch the report from the URL obtained by step 3.
- While using our sample code and tools
2005: YouTube is born
Me at the Zoo is the first video uploaded to YouTube
2006: Google buys YouTube
One year after YouTube launches, videos play in the FLV container with the H.263 codec at a maximum resolution of 240p. We scale videos up to 640x360, but you can still click a button to play at original size.
2007: YouTube goes mobile
YouTube is one of the original applications on the iPhone. Because it doesn't support Flash, we re-encode every single YouTube video into H.264 with the MP4 container. YouTube videos get a resolution notch to 360p.
2008: YouTube kicks it up to HD
With upload sizes and download speeds growing, videos jump in size up to 720p HD. Lower resolution files get higher quality by squeezing Main Profile H.264 into FLVs.
2009: YouTube enters the third dimension
YouTube supports 3D videos, 1080p and live streaming.
2010: YouTube's on TV
The biggest screen in your house now gets YouTube courtesy of Flash Lite and ActionScript 2. 2010 also sees the first playbacks with HTML5 <video> thanks to VP8, an open source video codec. We bump up the maximum resolution to 4K, known as "Original" at the time.
2011: YouTube slices bread (and videos) to battle buffering
We launch Sliced Bread, codename for a project that enables adaptive bitrate in the Flash player by requesting videos a little piece at a time. Users see higher quality videos more often and buffering less often.
2012: YouTube live streaming hits prime time
We scale up our live streaming infrastructure to support the 2012 Summer Olympics, with over 1,200 events. In October, over 8 million people watch live as Felix Baumgartner jumps from the stratosphere.
2013: YouTube's first taste of VP9
We start our first experiments with VP9 in Chrome, which brings higher quality video at less bandwidth. Adaptive bitrate streaming in the HTML5 and Flash players moves to the DASH standard using both FMP4 and MKV video containers.
2014: Silky smooth 60fps comes to YouTube
High frame rate isn't just for games anymore: YouTube now supports videos that play in up to 60fps. Gangnam Style becomes the first YouTube video to break the MAX_INT barrier with more than 232 / 2 - 1 views.
2015: YouTube adds spherical video (look behind you!)
You can now upload videos that wrap 360 degrees around the viewer. Even 4K videos can play up to 60fps. HTML5 becomes the default YouTube web player.
With the recent additions of comments, captions, and RSS push notifications, the Data API v3 supports almost every feature scheduled to be migrated from the soon-to-be-turned-down Data API v2. The only remaining feature to be migrated is video flagging, which will launch in the coming days. The new API brings in many features from the latest version of YouTube, making sure your users are getting the best YouTube experience on any screen.
For a quick memory lane trip, in March 2014, we announced that the Data API v2 would be retired on April 20, 2015, and would be shut down soon thereafter. To help with your migration, we launched the migration guide in September 2014, and have also been giving you regular notices on v3 feature updates.
By late May, v2 API calls except for comments and captions will receive 410 Gone HTTP responses. You can test your application’s reaction to this response by pointing the application at eol.gdata.youtube.com instead of gdata.youtube.com. While you should migrate your app as soon as possible, these features will work in the Data API v2 until the end of July 2015 to avoid any outages.
How you can migrate
- Ibrahim Ulukaya, and the YouTube for Developers team
Ibrahim 2 hours ago