Google BigQuery gets several updates that give BigQuery the ability to handle arbitrarily large result sets, use window functions for advanced analytics, and cache query results.
In addition, BigQuery also getting new UI features, larger interactive quotas, and a new convenient tiered pricing scheme.
The UI improvements rolled out today, including including the ability validate a query using “Query validator”, and estimate its cost prior to running it with “cost estimator”, and abandonment.
“Previously you had to wait until its completion before starting another one. Now you have the option to abandon it, to start working on the next iteration of the query without waiting for the abandoned one through the new “abandonment” feature,” explains Google.
And, Google has also doubled the existing interactive query quotas to support larger workloads for all users, “from 200GB + 1 concurrent query, to 400 GB of concurrent queries + 2 additional queries of unlimited size,” informs Google.
Starting in July, Google is offering everyone new pricing options with data storage now costs “$0.08/GB/month.” Furthermore, in addition to the existing on-demand rate for interactive queries, customers with higher-volume usage will soon be able to opt in for tiered query pricing, for even better value.
Other features released today include:
- Large results let BigQuery process terabytes of data, but until today BigQuery could only output up to 128 MB of compressed data per query. You can run queries that return arbitrarily large numbers of rows and save them as a new table for follow-up analysis.
All results will be saved to the new specified table (or appended, if the table exists). In the updated web UI these options can be found under the new “Enable Options” menu.
To get this benefit, you should enable the new “–allow_large_results” flag when issuing a query job, and specify a destination table.
- Window functions (also known as “analytical functions”) take advantage of built-in functions like Rank and Partition to create sophisticated statistical analyses with far simpler SQL than before.
“These new functions for ranking, percentiles, and relative row navigation, give you different ways to rank results, explore distributions and percentiles, and traverse results without the need for a self join,” informs Google.
- Query caching enable BigQuery return a cached result for the recent queries that are re-run, when the underlying table is unchanged, providing more cost-effective analysis.
“Cached results are only returned for tables that haven’t changed since the last query, or for queries that are not dependent on non-deterministic parameters (such as the current time). Reading cached results is free, but each query still counts against the max number of queries per day quota,” google adds.
Query results are kept cached for 24 hours, on a best effort basis.
You can disable query caching with the new flag –use_cache in bq, or “useQueryCache” in the API. “This feature is also accessible with the new query options on the BigQuery Web UI,” google added.
In other cloud product, Google posted a few things involving privacy and security of students in the Google Apps for Education.
Google Apps for Education is not ad-supported. Google notes, that advertising is by default “turned off” Apps for Education. “So unless a school administrator chooses to turn ads on, students will not see any of them while using Google Apps services,” google added.
Students and schools own their data that “they put into our systems, and we believe it should stay that way,” google said, adding, “If they decides to no longer use Google we make it easy for them to take their data away. Most importantly, Google does not sell students’ — or any other users’ — information,” google said.
Student accounts are never linked with other consumer Google accounts “they may open.”