๋ณธ๋ฌธ ๋ฐ”๋กœ๊ฐ€๊ธฐ

2019/0521

[๋ฆฌ๋ˆ…์Šค] ํŒŒ์ผ ์ด๋ฆ„ ์—ฌ๋Ÿฌ๊ฐœ ๋™์‹œ์— ๋ณ€๊ฒฝ ๋น„์Šทํ•œ ๊ทœ์น™์— ํŒŒ์ผ๋ช… ๋™์‹œ์— ์ผ๊ด„ ๋ณ€๊ฒฝํ•˜๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. rename '๊ณตํ†ต๋‚ด์šฉ' '๋ณ€๊ฒฝํ•  ๊ณตํ†ต๋‚ด์šฉ' '๋ณ€๊ฒฝ๋Œ€์ƒ' ex) rename .txt test.txt * 2019. 5. 31.
Java Date ์ •๊ทœ์‹ ์ž๋ฐ” date(๋‚ ์งœ) ํฌ๋งท ๋ณ„ ์ •๊ทœ์‹์ž…๋‹ˆ๋‹ค. public static final String YYYYMMDD = "(19|20)\\d{2}(0[1-9]|1[012])(0[1-9]|[12][0-9]|3[01])"; public static final String YYYYMMDDHH = "(19|20)\\d{2}(0[1-9]|1[012])(0[1-9]|[12][0-9]|3[01])(0[0-9]|1[0-9]|2[0-3])"; public static final String YYYYMMDDHHMI = "(19|20)\\d{2}(0[1-9]|1[012])(0[1-9]|[12][0-9]|3[01])(0[0-9]|1[0-9]|2[0-3])([0-5][0-9])"; public static final String YY.. 2019. 5. 31.
[๋ฆฌ๋ˆ…์Šค] jar ํŒŒ์ผ ์ƒ์„ฑ ๋ฐ ์‹คํ–‰ ๋ฐฉ๋ฒ• ์ดํด๋ฆฝ์Šค๋กœ jar ํŒŒ์ผ ์ƒ์„ฑํ•˜์—ฌ ๋ฆฌ๋ˆ…์Šค์ƒ์—์„œ jar ํŒŒ์ผ ์‹คํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. ์ž๋ฐ”์ฝ”๋“œ๋Š” ๋ฉ”์ธํ•จ์ˆ˜๋กœ ์ž…๋ ฅ๋˜๋Š” ๊ฐ’์„ ์ถœ๋ ฅํ•˜๋Š” ๊ฐ„๋‹จํ•œ ์ฝ”๋“œ๋กœ ์ž‘์„ฑํ–ˆ์Šต๋‹ˆ๋‹ค. public class test{ public static void main(String[] args){ for(String arg:args){ System.out.println(arg); } } } 1. eclipse๋กœ jar ํŒŒ์ผ ์ƒ์„ฑ ๋ฉ”์ธ ํด๋ž˜์Šค๋ฅผ ์ง€์ •ํ•ด์ฃผ์ง€ ์•Š์œผ๋ฉด no main manifest attribute, in test.jar ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค. 2. ์ƒ์„ฑํ•œ jar ํŒŒ์ผ์„ ๋ฆฌ๋ˆ…์Šค ์„œ๋ฒ„๋กœ ์ด๋™์‹œํ‚ค๊ณ  ์•„๋ž˜ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ java -jar test.jar TEST [์‹คํ–‰ ๊ฒฐ๊ณผ] TEST 2019. 5. 29.
[Hadoop] missing block ์ฐพ๋Š” ๋ฐฉ๋ฒ• โ—พ missing block error msg cannot obtain block length for LocatedBlock, Could not find CoarseGrainedScheduler or it has been stopped. โ—พ check corrupt file hdfs fsck -list-corruptfileblocks hdfs fsck /{path} -openforwrite 2019. 5. 29.
[Hive] parquet ์••์ถ• ์„ค์ • CREATE TABLE test(a int, b string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS PARQUET TBLPROPERTIES ("parquet.compression"="SNAPPY"); 2019. 5. 29.
[Hive] ์ฟผ๋ฆฌ๊ฒฐ๊ณผ ํŒŒ์ผ ์ถ”์ถœ โ—พ hive -e ๋ช…๋ น์–ด ์‚ฌ์šฉํ•˜์—ฌ ํŒŒ์ผ ์ €์žฅํ•˜๋Š” ๋ฐฉ๋ฒ• - beeline์œผ๋กœ select ๊ฒฐ๊ณผ๋ฅผ ์ถœ๋ ฅํ•  ๊ฒฝ์šฐ ์ปฌ๋Ÿผ๋ช…๊ณผ ํ…Œ์ด๋ธ” ํ˜•ํƒœ๋„ ๊ฐ™์ด ๋‚˜์˜ค๊ธฐ ๋•Œ๋ฌธ์—, outputformat=tsv2 ์˜ต์…˜์œผ๋กœ ์ œ๊ฑฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. hive --outputformat=tsv2 -e "select * from db.tb" > /local_path/test.dat โ—พ delimiter ๋ณ€๊ฒฝํ•˜์—ฌ ํŒŒ์ผ ์ €์žฅํ•˜๋Š” ๋ฐฉ๋ฒ• (hive to file) - ์•„๋ž˜๋‚ด์šฉ์—์„œ ์ €์žฅํ•  ํŒŒ์ผ์˜ ๋””๋ ‰ํ† ๋ฆฌ ์œ„์น˜์™€ ์›ํ•˜๋Š” ๊ตฌ๋ถ„์ž, ์ฟผ๋ฆฌ ์„ค์ • ํ›„ hql ํŒŒ์ผ ์ €์žฅ INSERT OVERWRITE LOCAL DIRECTORY './directory_name' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS .. 2019. 5. 29.
[์ดํด๋ฆฝ์Šค] Synchronize ํƒญ ํ•˜๋‹จ์— ๊ณ ์ •ํ•˜๊ธฐ ์ดํด๋ฆฝ์Šค์—์„œ Synchronize ์‚ฌ์šฉํ•  ๋•Œ ์ดˆ๊ธฐ ์„ค์ •์ด ํ™”๋ฉด ์ „ํ™˜๋˜๋„๋ก ์„ค์ •๋˜์–ด ์žˆ๋Š”๋ฐ ์ž‘์—…ํ•˜๋‹ค ๋ณด๋ฉด ์ƒ๋‹นํžˆ ๋ถˆํŽธํ•ฉ๋‹ˆ๋‹ค. ์•„๋ž˜ ์˜ต์…˜ ์ ์šฉํ•ด ์ฃผ๋ฉด ์•„๋ž˜ ์˜์—ญ์— ๊ณ ์ •๋ผ์„œ ํ™”๋ฉด ์ „ํ™˜๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. Window > Preferences > Team > Open the associated perspective when a synchronize operation completes > Never ํด๋ฆญ [์„ค์ • ํ›„ ์ ์šฉ ํ™”๋ฉด] 2019. 5. 24.
[์ดํด๋ฆฝ์Šค] ๊ธ€๊ผด ๋ฐ ํฌ๊ธฐ ๋ณ€๊ฒฝ ๋ฐฉ๋ฒ• eclipse ๊ธ€๊ผด ๋ฐ ํฌ๊ธฐ ๋ณ€๊ฒฝ ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. 1. Window -> Prefernces 2. General -> Appearance -> Colors and Fonts ->Basic -> Text Font -> Edit 3. ์›ํ•˜๋Š” ๊ธ€๊ผด ๋ฐ ํฌ๊ธฐ ์„ ํƒ 4. Apply and Close ํด๋ฆญํ•˜๋ฉด ๋ณ€๊ฒฝ ์™„๋ฃŒ 2019. 5. 24.