HDFS commands in HortonWorks Sandbox

This post is about how to use HDFS commands in a hortonwork sandbox.

If you have a file in local system and you like to move it to the hdfs.

Open Winscp with your sandbox hostname.
Go to /tmp directory.
Move the file from local system to the /tmp folder.
Check -ls /tmp to check whether the file is in the directory.

Now, you like to move to hdfs.

Put
 Copy file or files from local system into hdfs.
 (e.g)
 hdfs dfs -put /tmp/sample_test.csv  /user/jpraveen/sample_test.csv
Get
 Copy file or files from HDFS to local file system.
 (e.g)
 hdfs dfs -get /user/jpraveen/sample_test.csv /tmp/sample_test.csv
Move file from Local
 Moves file from local directory into cluster and deletes local copy.
 (e.g)
 hdfs dfs -moveFromLocal /tmp/sample_test.csv /user/jpraveen/sample_test.csv
Cat
 hdfs dfs -cat /user/jpraveen/sample_test.csv
mkdir
 Create a directory
 hdfs dfs -mkdir /user/jpraveen/hadoop-course
chmod
 hdfs dfs -chmod -R 700 /user/hadoop-course/inside
chgrp
 Change the group recursive
 hdfs dfs -chgrp -R jpraveen /user/hadoop-course/inside
chmod
 Change the ownership recursive
 hdfs dfs -chown -R hadoop /user/hadoop-course/inside
rm
 Remove the file
 hdfs dfs -rm /user/jpraveen/sample_test.csv
Recursive Remove
 You can also recursive remove the directory.
 hdfs dfs -rm -r /user/jpraveen

But it won’t entirely delete it. You can check those in trash folder.

Trash Folder
 If you remove the file it goes to the trash folder. To check that
 [jpraveen@sandbox ~]$ hdfs dfs -ls .Trash/Current/user
 Found 1 items
 drwx------ - jpraveen hdfs 0 2016-07-02 14:54 .Trash/Current/user/jpraveen
Empty the trash
 If you like to empty the trash use -expunge.
 hdfs dfs –expunge
 [jpraveen@sandbox ~]$ hdfs dfs -ls .Trash/Current/user
 ls: `.Trash/Current/user': No such file or directory.

Please comment if you have any doubts/questions.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s