Feeds:
Posts
Comments

If one launches Hive using the “-S” or “–silent” option then Hive does not print progress information. However if you are already inside the Hive command line shell then you can control this behaviour by setting the value of silent.


set silent=on; -- make hive silent

set silent=false; -- make hive print progress information

 

Advertisements

window functions allow one to look at the previous values or next values of a column. for example if i want to subtract the previous row value from the current row value then window functions lag and lead can be used.

let us take up an example.

first create a text file containing numbers 1 through 9 with a single number on each line like so


1

2

3

.

.

9

call the file data.txt.

next we create a table in hive.


create table foo (a int);

next we load our data.txt file in to this created table using the following


load data local inpath 'data.txt' overwrite into table foo;

we want to access previous and next values over column ‘a’ note therefore the over clause in the following query


select lag(a, 1) over (order by a) as previous, a, lead(a, 1) over (order by a) as next from foo;

which outputs the following:


previous a next
NULL 1 2
1 2 3
2 3 4
3 4 5
4 5 6
5 6 7
6 7 8
7 8 9
8 9 NULL

note how the previous and next values are NULL at the edge cases. You could specify a value in such cases as in the following query which specifies 0.


select lag(a, 1, 0) over (order by a) as previous, a, lead(a, 1, 0) over (order by a) as next from foo;

which outputs the following:


previous a next
0 1 2
1 2 3
2 3 4
3 4 5
4 5 6
5 6 7
6 7 8
7 8 9
8 9 0

lag(a, 1) will fetch the previous value while lag(a, 2) will fetch the previous to previous value.

case matters.

had created table as such

create table foo(

a int,

b string

) stored as orc tblproperties(“orc.compress”=”snappy”);

but when i went to populate the table using an “insert overwrite table foo select * from …” statement then i faced an error.

turns out i should have used “SNAPPY” instead of “snappy”. case matters.

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC

if you do not want to apply markdown formatting rules to a chunk of text then wrap them in following tags.

opening tag: “`text i.e. 3 backtics followed by the word text

closing tag: “` 3 backticks

this will give you unformatted text and displays it in exactly the form it was typed in.

import datetime

print(datetime.datetime.strftime(datetime.datetime.now(), ‘%c’))

hope this becomes the number 1 search result when people type this post’s title in to google.

i like to sort my vifm buffers by modification time but don’t like the fact that all the directories are grouped first and then the files next. i wanted to sort the buffer so that there is no distinction between directories and files when sorting by modification time. here is how you can achieve it.

issue the colon(:) to access the vifm command prompt

then issue the following command

set sort=mtime,dir

mtime is for modification time and dir for directories.

i have mapped that to the leader key+s combo in vifmrc file thusly

nmap ,s :set sort=mtime,dir<cr>

matplotlib default figsize

The default value is [8.0, 6.0] which can be changed of course. To know all the default values just inspect the value of ‘rcParams


import matplotlib.pyplot as plt

print(plt.rcParams) # to examine all values

print(plt.rcParams.get('figure.figsize')