menu

Batch Load Data Into BigQuery

Go to Lab

173 Reviews

B67c5fea4a4534182937f2f61f35b889

Sunil B. · Reviewed about 15 hours ago

2ba362dfc2e50300291bd6cfa4b8b03a

okay. it would be great to review the initial dataset from Wikipedia

Laurent B. · Reviewed 2 days ago

44f378712e6d586d0d4d992e4937d6d2

Daniel S. · Reviewed 2 days ago

678c0470ba17e1cb44646fbc688e9b3f

Daniel H. · Reviewed 3 days ago

A6988df6d7ab83529ee7870844da2554

Hugo A. · Reviewed 4 days ago

66a059361c7721e33d208d7a3109524f

Thanks

Yandi S. · Reviewed 5 days ago

E8425d1ec83e660f365ceb7adc58cd20

Iain K. · Reviewed 6 days ago

7e199d504dce9848825562de89d900a6

Samuel T. · Reviewed 6 days ago

427a7bfc092eb717e2c9ab741e14d960

Simplistic

Christopher T. · Reviewed 6 days ago

E0fd722be5f41cb819b7f293719d7a36

Mihail I. · Reviewed 6 days ago

9d398d1f9e7ee5f7885d3580ce128939

Alex H. · Reviewed 7 days ago

Fc50a55e4760ec356346067b083afcd1

John B. · Reviewed 8 days ago

B3630baad8ffd42bfd460fde12e6c9bd

Welcome to Cloud Shell! Type "help" to get started. google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ wget https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160101-070000.gz --2017-12-02 17:30:16-- https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160101-070000.gz Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 208.80.154.11, 2620:0:861:1:208:80:154:11 Connecting to dumps.wikimedia.org (dumps.wikimedia.org)|208.80.154.11|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 73747078 (70M) [application/octet-stream] Saving to: ‘pagecounts-20160101-070000.gz’ pagecounts-20160101-070000.gz 100%[==============================================================================================================================================================>] 70.33M 1.99MB/s in 37s 2017-12-02 17:30:53 (1.92 MB/s) - ‘pagecounts-20160101-070000.gz’ saved [73747078/73747078] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_11 \ pagecounts_20160101-070000.gz \ language,title,requests:integer,content_size:integer Welcome to BigQuery! This script will walk you through the process of initializing your .bigqueryrc configuration file. First, we need to set up your credentials if they do not already exist. Credential creation complete. Now we will select a default project. List of projects: # projectId friendlyName --- ------------------------------- ------------------------------- 1 qwiklabs-resources Qwiklabs Resources 2 qwiklabs-gcp-27cb66854bc10d96 qwiklabs-gcp-27cb66854bc10d96 Found multiple projects. Please enter a selection for which should be the default, or leave blank to not set a default. Enter a selection (1 - 2): 2 BigQuery configuration complete! Type "bq" to get started. Too many positional args, still have [' pagecounts_20160101-070000.gz', ' language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ > --quote "" \ > YourProjectID:lab.pagecounts_20160108_11 \ > pagecounts-20160108-110000.gz \ > BigQuery error in load operation: Source file not found: pagecounts-20160108-110000.gz google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " "\ --quote ""\ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \ pagecounts-20160101-070000.gz \ language,title,requests:integer,content_size:integer BigQuery error in load operation: Field delimiter must be a single character, found:" --quote". google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \ pagecounts-20160101-070000.gz \ language,title,requests:integer,content_size:integer Too many positional args, still have [' pagecounts-20160101-070000.gz', ' language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \pagecounts-20160101-070000.gz \ language,title,requests:integer,content_size:integer Too many positional args, still have ['pagecounts-20160101-070000.gz', ' language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \pagecounts-20160101-070000.gz \language,title,requests:integer,content_size:integer Too many positional args, still have ['pagecounts-20160101-070000.gz', 'language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ wget https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160103-040000.gz --2017-12-02 17:48:03-- https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160103-040000.gz Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 208.80.154.11, 2620:0:861:1:208:80:154:11

John C. · Reviewed 8 days ago

B3630baad8ffd42bfd460fde12e6c9bd

Welcome to Cloud Shell! Type "help" to get started. google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ wget https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160101-070000.gz --2017-12-02 17:30:16-- https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160101-070000.gz Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 208.80.154.11, 2620:0:861:1:208:80:154:11 Connecting to dumps.wikimedia.org (dumps.wikimedia.org)|208.80.154.11|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 73747078 (70M) [application/octet-stream] Saving to: ‘pagecounts-20160101-070000.gz’ pagecounts-20160101-070000.gz 100%[==============================================================================================================================================================>] 70.33M 1.99MB/s in 37s 2017-12-02 17:30:53 (1.92 MB/s) - ‘pagecounts-20160101-070000.gz’ saved [73747078/73747078] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_11 \ pagecounts_20160101-070000.gz \ language,title,requests:integer,content_size:integer Welcome to BigQuery! This script will walk you through the process of initializing your .bigqueryrc configuration file. First, we need to set up your credentials if they do not already exist. Credential creation complete. Now we will select a default project. List of projects: # projectId friendlyName --- ------------------------------- ------------------------------- 1 qwiklabs-resources Qwiklabs Resources 2 qwiklabs-gcp-27cb66854bc10d96 qwiklabs-gcp-27cb66854bc10d96 Found multiple projects. Please enter a selection for which should be the default, or leave blank to not set a default. Enter a selection (1 - 2): 2 BigQuery configuration complete! Type "bq" to get started. Too many positional args, still have [' pagecounts_20160101-070000.gz', ' language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ > --quote "" \ > YourProjectID:lab.pagecounts_20160108_11 \ > pagecounts-20160108-110000.gz \ > BigQuery error in load operation: Source file not found: pagecounts-20160108-110000.gz google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " "\ --quote ""\ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \ pagecounts-20160101-070000.gz \ language,title,requests:integer,content_size:integer BigQuery error in load operation: Field delimiter must be a single character, found:" --quote". google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \ pagecounts-20160101-070000.gz \ language,title,requests:integer,content_size:integer Too many positional args, still have [' pagecounts-20160101-070000.gz', ' language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \pagecounts-20160101-070000.gz \ language,title,requests:integer,content_size:integer Too many positional args, still have ['pagecounts-20160101-070000.gz', ' language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ bq load -F " " \ --quote "" \ qwiklabs-gcp-27cb66854bc10d96:lab.pagecounts_20160108_07 \pagecounts-20160101-070000.gz \language,title,requests:integer,content_size:integer Too many positional args, still have ['pagecounts-20160101-070000.gz', 'language,title,requests:integer,content_size:integer'] google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ google151245_student@qwiklabs-gcp-27cb66854bc10d96:~$ wget https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160103-040000.gz --2017-12-02 17:48:03-- https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-01/pagecounts-20160103-040000.gz Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 208.80.154.11, 2620:0:861:1:208:80:154:11

John C. · Reviewed 8 days ago

6d043aee4e216be85d2502057f483cf7

Brian B. · Reviewed 9 days ago

1a4e894b904555b9567a784038dd8428

Cyrille H. · Reviewed 10 days ago

Fda62f7732261726a6fcde9ca3cd18e9

Ricquan W. · Reviewed 10 days ago

C91495c0b28fcab82218ffadc9ca3a6a

Takes 10 minutes to upload the file.

Ben A. · Reviewed 10 days ago

Fda62f7732261726a6fcde9ca3cd18e9

Ricquan W. · Reviewed 10 days ago

Bfc5c85065974c0a7e35e77d933e832f

Christian M. · Reviewed 11 days ago

75c67993957bc996c1450607ab75d710

This lab doesn't teach much more then where the buttons are. An explanation of the command of loading the data into BQ would have been a good start.

Jonathan Y. · Reviewed 11 days ago

41100877d5ede92728dfba22716edd55

Greg F. · Reviewed 13 days ago

19b5e626ebd1a9ddeb068fa7a7530fee

Ignas G. · Reviewed 13 days ago

19b5e626ebd1a9ddeb068fa7a7530fee

Ignas G. · Reviewed 13 days ago

9b9d7df2f74ff68fa1b96f009fcb9507

Elvina A. · Reviewed 13 days ago