๐ปTech159 Docker custom image ์์ฑ ๋ฐฉ๋ฒ (with CDSW) 1. Dockerfile ์์ฑ $ vi Dockerfile ------------------------------------------------------------------- # Dockerfile # Specify a Cloudera Data Science Workbench base image FROM docker.repository.cloudera.com/cdsw/engine:13 # Update packages on the base image and install beautifulsoup4 RUN pip3 install beautifulsoup4 ------------------------------------------------------------------- 2. Dockerfile b.. 2021. 1. 8. kudu ๋ช ๋ น์ด ์ ๋ฆฌ โพ ํ ์ด๋ธ ์์ฑ (as select) CREATE TABLE temp.kudu_cm_audits PRIMARY KEY (audit_id) STORED AS KUDU TBLPROPERTIES ('kudu.num_tablet_replicas' = '1') AS SELECT * FROM temp.cm_audits ; โพ ํ ์ด๋ธ ์์ฑ (range partition) CREATE TABLE IF NOT EXISTS temp.kudu_sample( gubun string, seq int, code string, description string, total_emp int, salary int, PRIMARY KEY(gubun,seq) ) PARTITION BY RANGE(gubun)( PARTITION VALUE .. 2020. 12. 23. [NiFi] Impala JDBC DBCPConnectionPool ์ฌ์ฉ ์์ 1. Impala JDBC Connector ๋ค์ด๋ก๋ https://www.cloudera.com/downloads/connectors/impala/jdbc/2-6-17.html 2. NiFi DBCPConnectionPool ์ค์ Database Driver Location(s) ์ ๋ก๋ํ ํ์ผ ๊ฒฝ๋ก ์ ๋ ฅ /opt/cloudera/parcels/CFM/NIFI/lib/ImpalaJDBC42.jar ๊ฒฝ๋ก๋ฅผ ์ง์ ํ์ง ์์ ๊ฒฝ์ฐ ์๋์ ๊ฐ์ ์๋ฌ๊ฐ ๋ฐ์ํจ Validation query ์ ๋ ฅํด์ฃผ๋ฉด ์ข์ (select 1) Could not initialize class com.cloudera.impala.jdbc42.internal.apache.thrift.transport.TSocket 3. Impala SQ.. 2020. 10. 14. ์ฟ ๋ฒ๋คํฐ์ค ๋ช ๋ น์ด (with CDSW) โพ ์ฟ ๋ฒ๋คํฐ์ค pod ๋ฆฌ์คํธ ์กฐํ kubectl get pods --all-namespaces -o wide โพ pod ์ ์ kubectl exec -it ํ์ด๋ฆ -n ๋ค์์คํ์ด์ค /bin/bash ex) kubectl exec -it 5w1ixw5q39iqvcfi -n default-user-1 /bin/bash โพ pod ๋ก๊ทธ ํ์ธ ๋ฐฉ๋ฒ ## f ์ต์ ์ ์ค์๊ฐ ํ์ธ kubectl logs -f 5w1ixw5q39iqvcfi -n default-user-1 ## pod init ๋ก๊ทธ ํ์ธ (Init:CrashLoopBackOff, Init:Error ๋ฐ์ํ ๋ ํ์ธ) ## init-container name์ describe ๋ช ๋ น์ด ์คํ์ผ๋ก ํ์ธ ๊ฐ๋ฅ kubectl logs 5w1ixw5q39iqvcf.. 2020. 10. 13. Cloudera Streams Messaging Manager Installation ๐ซ ERROR org.flywaydb.core.api.FlywayException: Unable to instantiate JDBC driver class not found exception com.mysql.cj.jdbc.driver ๐ก SOLVED https://docs.cloudera.com/cdp-private-cloud-base/latest/installation/topics/csp-configuring-schema-registry-metadata-stores-in-mysql.html 2020. 9. 17. Websocket connection failed ws:// err_name_not_resolved ๐ซ ERROR Websocket connection failed ws://ex.websocket.com err_name_not_resolved ๐ก SOLVED ์ ์ํ๋ PC์ hosts ํ์ผ์ ํด๋น ํธ์คํธ๋ช ๊ณผ IP๋ฅผ ์ ๋ ฅํด์ค๋๋ค. C:\Windows\System32\drivers\etc\hosts ํ์ผ ์ด์ด์ ์๋ ๋ด์ฉ ์ ๋ ฅ 128.x.x.1 ex.websocket.com 2020. 9. 10. OpenLDAP + phpLDAPadmin ์ค์น ๋ฐ ์ค์ OpenLDAP LDAP ํ๋กํ ์ฝ์ ์ฌ์ฉํ์ฌ ๋๋ ํ ๋ฆฌ(์ฌ์ฉ์/๊ทธ๋ฃน) ์ ๋ณด๋ฅผ ์ ์ฅํ๊ณ ๊ฒ์ํ๋ ์ญํ ์ ์๋น์ค phpLDAPadmin OpenLDAP ์๋ฒ๋ฅผ ์น ๊ธฐ๋ฐ์ผ๋ก ๊ด๋ฆฌํ๊ธฐ ์ํ LDAP ํด๋ผ์ด์ธํธ ์น ์ ํ๋ฆฌ์ผ์ด์ 1. Install OpenLDAP $ sudo yum -y install openldap-servers openldap-clients $ sudo systemctl start slapd $ sudo systemctl status slapd $ sudo systemctl enable slapd $ sudo yum -y install net-tools $ sudo netstat -antup | grep -i 389 tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN 8969/sl.. 2020. 9. 2. [SOLVED] UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 80: ordinal not in range(128) ๐ซ ERROR hue์์ hive ์ฌ์ฉํ ๋ ํ๊ธ์ insert ํ๋ ค๋ฉด ์ธ์ฝ๋ฉ ๊ด๋ จ ์๋ฌ๊ฐ ๋ฐ์ํ๋๋ฐ์. UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 80: ordinal not in range(128) ๐ก SOLVED hue์์ ์ฟผ๋ฆฌ๋ฅผ ์ฒ๋ฆฌํ ๋ ์คํ๋๋ python ์ธ์ฝ๋ฉ ํ์ ์ด ascii๋ก ์ค์ ๋์ด ์์ด์ ๊ทธ๋ ์ต๋๋ค. ํด๋น ํ์ด์ฌ ์คํฌ๋ฆฝํธ ์ฐพ์์ ์์ ํ Hue ์ฌ๊ธฐ๋ ํ๋ฉด ๋ฉ๋๋ค. (ascii > utf-8) vi /opt/cloudera/parcels/CDH/lib/hue/build/env/lib/python2.7/site.py ----------------------------------------------------.. 2020. 7. 6. ์ด์ 1 ยทยทยท 4 5 6 7 8 9 10 ยทยทยท 20 ๋ค์