๐Ÿ’ป 155

Apache Cassandra

๊ฐœ์š” ์•„ํŒŒ์น˜ ์นด์‚ฐ๋“œ๋ผ(Apache Cassandra)๋Š” ์˜คํ”ˆ ์†Œ์Šค๋กœ ๊ฐœ๋ฐœ๋œ ๋ถ„์‚ฐํ˜• NoSQL ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค์ž…๋‹ˆ๋‹ค. ์ด๋Š” ํŒŒํ‹ฐ์…˜๋œ ๋„“์€ ์—ด(column) ์ €์žฅ ๋ชจ๋ธ์„ ์ œ๊ณตํ•˜๋ฉฐ, ์ตœ์ข… ์ผ๊ด€์„ฑ(eventual consistency)์„ ๊ฐ€์ง„๋‹ค. ์•„ํŒŒ์น˜ ์นด์‚ฐ๋“œ๋ผ๋Š” ์›๋ž˜ Facebook์—์„œ ์„ค๊ณ„๋˜์—ˆ์œผ๋ฉฐ, staged event-driven architecture (SEDA)๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๊ตฌํ˜„๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ์ด๋Š” Amazon์˜ Dynamo ๋ถ„์‚ฐ ์ €์žฅ ๋ฐ ๋ณต์ œ ๊ธฐ์ˆ ๊ณผ Google์˜ Bigtable ๋ฐ์ดํ„ฐ ๋ฐ ์ €์žฅ ์—”์ง„ ๋ชจ๋ธ์˜ ๊ฒฐํ•ฉ์„ ๊ตฌํ˜„ํ•˜๊ธฐ ์œ„ํ•ด ๋งŒ๋“ค์–ด์กŒ์Šต๋‹ˆ๋‹ค. Dynamo์™€ Bigtable์€ ๋ชจ๋‘ ํ™•์žฅ ๊ฐ€๋Šฅํ•˜๊ณ  ์‹ ๋ขฐ์„ฑ์ด ๋†’์œผ๋ฉฐ ๊ณ ๊ฐ€์šฉ์„ฑ์„ ๊ฐ–์ถ˜ ์ €์žฅ ์‹œ์Šคํ…œ์˜ ์š”๊ตฌ ์‚ฌํ•ญ์„ ์ถฉ์กฑ์‹œํ‚ค๊ธฐ ์œ„ํ•ด ๊ฐœ๋ฐœ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ๊ฐ๊ฐ์€ ๊ฐœ์„ ํ•  ..

๐Ÿ’ป/MLOps 2023.07.31

.yaml ์—์„œ ์ค„ ๋ฐ”๊ฟˆํ•˜๋Š” 9๊ฐ€์ง€ ๋ฐฉ๋ฒ•

https://stackoverflow.com/questions/3790454/how-do-i-break-a-string-in-yaml-over-multiple-lines How do I break a string in YAML over multiple lines? I have a very long string: Key: 'this is my very very very very very very long string' I would like to express it over multiple shorter lines, e.g., Key: 'this is my very very very ' + 'long ... stackoverflow.com

[Pinot] ์˜คํ”ˆ์†Œ์Šค ๊ธฐ์—ฌ ํ”„๋กœ์„ธ์Šค

1. ๋“ค์–ด๊ฐ€๊ธฐ ์ „, 1.1 ๊ฐœ๋ฐœ ํ™˜๊ฒฝ ์„ค์ • [Pinot] Apache Pinot ๊ฐœ๋ฐœ ํ™˜๊ฒฝ Apache Pinot ์šด์˜์ด ์•„๋‹Œ, ๊ฐœ๋ฐœ ๋ฐ ํ”„๋กœ์ ํŠธ ๊ธฐ์—ฌ๋ฅผ ๋ชฉ์ ์œผ๋กœ ๊ฐœ๋ฐœ ํ™˜๊ฒฝ์„ ์„ค์ •ํ•œ๋‹ค. WSL ํ™˜๊ฒฝ์„ ๊ธฐ์ค€์œผ๋กœ ํ•œ๋‹ค. Running Pinot locally - Apache Pinot Docs Note that some installations of the JDK do not contain the JNI binding hhlab.tistory.com 1.2 ๋ชจ๋“ˆ ๋ฐ ์•„ํ‚คํ…์ณ pinot-common ํ”„๋กœ์ ํŠธ ์ „๋ฐ˜์˜ ๊ณตํ†ต ํด๋ž˜์Šค๋“ค์„ ์ œ๊ณต (config, metrics, metadata, pql.parsers, request, response, fs, ...) pinot-trasnport Broker์˜ ๋ถ„์‚ฐ-์ˆ˜์ง‘..

๐Ÿ’ป/MLOps 2023.03.05

[Pinot] Apache Pinot ๊ฐœ๋ฐœ ํ™˜๊ฒฝ

Apache Pinot ์šด์˜์ด ์•„๋‹Œ, ๊ฐœ๋ฐœ ๋ฐ ํ”„๋กœ์ ํŠธ ๊ธฐ์—ฌ๋ฅผ ๋ชฉ์ ์œผ๋กœ ๊ฐœ๋ฐœ ํ™˜๊ฒฝ์„ ์„ค์ •ํ•œ๋‹ค. WSL ํ™˜๊ฒฝ์„ ๊ธฐ์ค€์œผ๋กœ ํ•œ๋‹ค. Running Pinot locally - Apache Pinot Docs Note that some installations of the JDK do not contain the JNI bindings that are necessary to run all tests, if you see any java.lang.UnsatisfiedLinkError while running tests, you may need to change your JDK. If using Homebrew, you may install AdoptOpenJDK docs.pinot.apache.org 1. Java..

๐Ÿ’ป/MLOps 2023.03.05

[Pinot] Apache Pinot?

Apache Pinot์˜ ๊ณต์‹ ๋ฌธ์„œ๋ฅผ ์š”์•ฝํ•œ ๊ธ€์ž…๋‹ˆ๋‹ค. https://docs.pinot.apache.org/ "Apache Pinot์€ ๋น ๋ฅธ ์‘๋‹ต์†๋„๋กœ ๋งŽ์€ ๋ถ„์„์„ ์ œ๊ณตํ•  ์ˆ˜ ์žˆ๋Š”, ์‹ค์‹œ๊ฐ„ ๋ถ„์‚ฐ OLAP ๋ฐ์ดํ„ฐ ์ €์žฅ์†Œ" Kafka, Kinesis ๊ฐ™์€ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ ์†Œ์Šค๋“ค์—์„œ ๋‹ค์ด๋ ‰ํŠธ๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ๊ณ , ์ฆ‰์‹œ ์ฟผ๋ฆฌ ์‹คํ–‰์„ ์œ„ํ•œ ์ด๋ฒคํŠธ๋ฅผ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•œ๋‹ค. ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฟ ์•„๋‹ˆ๋ผ Hadoop, S3, ADLS, GCS์™€ ๊ฐ™์€ ๋ฐฐ์น˜ ๋ฐ์ดํ„ฐ ์†Œ์Šค์—์„œ๋„ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ๋‹ค. ์ด ์‹œ์Šคํ…œ์˜ ์ค‘์‹ฌ ๋ถ€์—๋Š”, ๋น ๋ฅธ ์‘๋‹ต ์†๋„๋ฅผ ์œ„ํ•œ ์Šค๋งˆํŠธ ์ธ๋ฑ์‹ฑ, ๋ฐ์ดํ„ฐ ์‚ฌ์ „ ์ง‘๊ณ„ ๊ธฐ๋Šฅ์„ '๊ณ๋“ค์ธ' ์ปฌ๋Ÿผ ์ง€ํ–ฅ ์ €์žฅ์†Œ๊ฐ€ ์žˆ๋‹ค. ์ด ์ ์ด Pinot์„ ์‹ค์‹œ๊ฐ„ ๋ถ„์„ ์ œ๊ณต์— ์™„๋ฒฝํ•œ ๋„๊ตฌ๋กœ ๋งŒ๋“ค์–ด์ค€๋‹ค. ๋ฟ๋งŒ ์•„๋‹ˆ๋ผ, Pinot์€ ๋‚ด๋ถ€ ๋Œ€์‹œ๋ณด๋“œ,..

๐Ÿ’ป/MLOps 2022.11.05

[๋ธ”๋ก์ฒด์ธ] quorum-dev-quickstart ์—ฐ๋™ ํ…Œ์ŠคํŠธ

Windows WSL2 ๊ธฐ์ค€์œผ๋กœ ์ง„ํ–‰ 1. Node.js ์„ค์น˜ # 1. cURL ์„ค์น˜ sudo apt-get install curl # 2. nvm ์„ค์น˜(Node.js Version Manager) curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/master/install.sh # 3. node.js ์„ค์น˜ nvm install --lts 2. docker desktop ์‹คํ–‰ docker --version 3. truffle ์„ค์น˜ npm install truffle -g 4. quorum-dev-quickstart ๋‹ค์šด๋กœ๋“œ npx quorum-dev-quickstart 5. web3js-quorum ์„ค์น˜ ์—ฐ๋™ํ•  ํ”„๋กœ์ ํŠธ์— web3js-quorum์„ ์„ค์น˜ํ•ด์ฃผ์ž..

[๋ธ”๋ก์ฒด์ธ] Windows ํ™˜๊ฒฝ์—์„œ Ganache ๋„คํŠธ์›Œํฌ ํ…Œ์ŠคํŠธ

Ganache? Ganache๋Š” ๊ฐ„์ด ์ด๋”๋ฆฌ์›€ ๋ธ”๋ก์ฒด์ธ์ด๋‹ค. Ganache๋ฅผ ์„ค์น˜ํ•˜๋ฉด, ๋กœ์ปฌ์—์„œ ๋ธ”๋ก์ฒด์ธ ๋„คํŠธ์›Œํฌ๊ฐ€ ๊ตฌ์„ฑ๋˜๊ณ , 100ETH๋ฅผ ๊ฐ€์ง„ 10๊ฐœ์˜ ํ…Œ์ŠคํŠธ ๊ณ„์ •์„ ๋ฐ›์„ ์ˆ˜ ์žˆ๋‹ค. ๊ฐœ๋ฐœ ์‹œ์— ๋น ๋ฅด๊ฒŒ ํ™˜๊ฒฝ์„ ๊ตฌ์„ฑํ•  ์ˆ˜ ์žˆ์œผ๋ฉฐ, ๋ฉ”๋ชจ๋ฆฌ ๋‚ด ๋ธ”๋ก์ฒด์ธ์ด๊ธฐ ๋•Œ๋ฌธ์— ํŠธ๋žœ์žญ์…˜ ํ…Œ์ŠคํŠธ ์‹œ ์‹ค์ œ ๋ธ”๋ก์ฒด์ธ ๋„คํŠธ์›Œํฌ์— ๋ฐฐํฌํ•˜๋Š” ๊ฒฝ์šฐ ๋ณด๋‹ค ์‹œ๊ฐ„์„ ํ›จ์”ฌ ์ ˆ์•ฝํ•  ์ˆ˜ ์žˆ๋‹ค. Ganache - Truffle Suite Features VISUAL MNEMONIC & ACCOUNT INFO Quickly see the current status of all accounts, including their addresses, private keys, transactions and balances. trufflesuite.com ๊ฐœ๋ฐœ..

[Backend] AJAX์˜ ์ดํ•ด

AJAX (Asynchronous Javascript And XML) JavaScript๋ฅผ ์ด์šฉํ•ด ์š”์ฒญ์„ ๋ณด๋‚ด๊ณ  ๋ฐ์ดํ„ฐ๋ฅผ ๋ฐ›์•„์™€ ํŽ˜์ด์ง€ ์ƒˆ๋กœ๊ณ ์นจ ์—†์ด, ํ™”๋ฉด์˜ ์ผ๋ถ€๋ฅผ ๊ฐฑ์‹ ํ•˜๋Š” ๋ฐฉ์‹์ด๋‹ค. ๊ฒฐ๊ตญ ๋ธŒ๋ผ์šฐ์ €๊ฐ€ ํ™”๋ฉด์„ ๋„์šฐ๊ธฐ ์œ„ํ•ด ํ•„์š”ํ•œ ๊ฒƒ์€ HTML/CSS์ด๋‹ค. Javascript๋กœ ํ™”๋ฉด์ด ๋ฐ”๋€Œ๋Š” ๊ฒƒ๋„ ๊ฒฐ๊ตญ DOM ๊ฐ์ฒด์— ์ ‘๊ทผํ•ด์„œ HTML ๋ฌธ์„œ๋ฅผ ์ˆ˜์ •ํ•˜๋Š” ๊ฒƒ์ด๋‹ค. Javascript๊ฐ€ ์š”์ฒญ์„ ๋ณด๋‚ด๊ณ  ๋ฐ›์„ ์ˆ˜ ์žˆ๋‹ค๋ฉด, ํ™”๋ฉด์˜ ํŠน์ • ๋ถ€๋ถ„๋งŒ์„ ์—…๋ฐ์ดํŠธ ํ•  ์ˆ˜ ์žˆ๋‹ค. ์ด ๋•Œ ์ด ์š”์ฒญ์— ๋Œ€ํ•œ ์‘๋‹ต์„ ๊ธฐ๋‹ค๋ ค์•ผ ํ•œ๋‹ค๋ฉด, ์‘๋‹ต์ด ์˜ฌ ๋•Œ ๊นŒ์ง€ ํ™”๋ฉด์„ ์ œ๋Œ€๋กœ ๋ณผ ์ˆ˜๊ฐ€ ์—†๊ธฐ ๋•Œ๋ฌธ์—, ๋‹น์—ฐํ•˜๊ฒŒ๋„ ๋น„๋™๊ธฐ์ ์œผ๋กœ ์š”์ฒญํ•ด์•ผ ํ•œ๋‹ค. ๋น„๋™๊ธฐ์ ์ด๋ž€ ์ผ์„ ์ฐจ๋ก€์ฐจ๋ก€ ์ฒ˜๋ฆฌํ•˜๋Š” ๊ฒƒ์ด ์•„๋‹ˆ๋ผ, ํŠน์ • ์—…๋ฌด๋ฅผ ๋‹ค๋ฅธ ๊ณณ์— ์˜๋ขฐํ•˜๊ณ , ๋๋‚œ ์ผ์„ ํƒ๋ฐฐํ•จ์— ๋†“๊ณ  ..

๐Ÿ’ป/์›น 2022.07.11

[Javascript] ๋น„๋™๊ธฐ ํ”„๋กœ๊ทธ๋ž˜๋ฐ(์‹ฑ๊ธ€ ์“ฐ๋ ˆ๋“œ, ๋„Œ-๋ธ”๋กœํ‚น)

Javascript๋Š” ์‹ฑ๊ธ€ ์Šค๋ ˆ๋“œ(Single Thread)์™€ ๋„Œ-๋ธ”๋กœํ‚น(Non-Blocking)์ด๋ผ๋Š” ๋‘ ๊ฐ€์ง€ ํŠน์„ฑ์„ ๊ฐ€์ง€๊ณ  ์žˆ๋‹ค. ์–ผํ• ๋ณด๋ฉด ์ด ๋‘๊ฐ€์ง€ ํŠน์„ฑ์€ ์–‘๋ฆฝ ๋ถˆ๊ฐ€๋Šฅํ•ด๋ณด์ธ๋‹ค. "์–ด๋–ป๊ฒŒ ๋™์‹œ์— ์—ฌ๋Ÿฌ ๊ฐœ์˜ ์ผ์„ ํ•  ์ˆ˜ ์—†์œผ๋ฉด์„œ, ํ•˜๋‚˜์˜ ์ผ์ด ๋๋‚  ๋•Œ ๊นŒ์ง€ ๊ธฐ๋‹ค๋ฆฌ์ง€ ์•Š๊ณ  ๋™์‹œ์— ์ฒ˜๋ฆฌํ•  ์ˆ˜ ์žˆ์ง€?" ๋‹ต์€ JS ์—”์ง„๊ณผ ๋ธŒ๋ผ์šฐ์ €์˜ ๋™์ž‘ ๊ตฌ์กฐ์— ์žˆ๋‹ค. JS ์—”์ง„ ์ž์ฒด๋Š” ์‹ฑ๊ธ€ ์“ฐ๋ ˆ๋“œ๋กœ ํ•œ ๊ฐ€์ง€ ์ผ๋งŒ ์ง‘์ค‘ํ•  ์ˆ˜ ์žˆ๋Š” ์นœ๊ตฌ๋‹ค. ๊ณ ๋ง™๊ฒŒ๋„ ์ด ๋‹จ์ˆœํ•œ ์นœ๊ตฌ๋ฅผ ์œ„ํ•ด ๋ธŒ๋ผ์šฐ์ €์—์„œ ์—ฌ๋Ÿฌ๊ฐ€์ง€ ๊ธฐ๋Šฅ์„ Web API๋ผ๋Š” ์ด๋ฆ„์œผ๋กœ ์ œ๊ณตํ•˜๊ณ  ์žˆ๋‹ค. JS ์—”์ง„์€ ๋™์‹œ์— ์ฒ˜๋ฆฌํ•ด์•ผํ•˜๋Š” ์ผ์„ Web API์— ์š”์ฒญํ•ด ๋†“๊ณ , ๋ณธ์ธ์ด ๋Š˜ ํ•˜๋˜๋Œ€๋กœ ํ•œ ๋ฒˆ์— ํ•˜๋‚˜์”ฉ๋งŒ ์ผ์„ ์ฒ˜๋ฆฌํ•˜๋Š” ๊ฒƒ์ด๋‹ค. Web API๋Š” ๋ณธ์ธ์ด ์ฒ˜๋ฆฌํ•œ ๊ฒฐ๊ณผ๋ฅผ Callback Qu..

๐Ÿ’ป/์›น 2022.07.11

[CI/CD] Vue ํ”„๋กœ์ ํŠธ๋ฅผ Jenkins (docker-in-docker) ๋ฐฐํฌํ•˜๊ธฐ

๐Ÿ“ƒ๊ฐœ์š” ๋„์ปค๋กœ Jenkins ์ปจํ…Œ์ด๋„ˆ๋ฅผ ์‹คํ–‰ํ•œ๋‹ค. Jenkins ์ปจํ…Œ์ด๋„ˆ์— docker-in-docker ๊ด€๋ จ ์„ธํŒ…์„ ํ•œ๋‹ค. GitLab ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์— ์ปค๋ฐ‹์ด ๋ฐœ์ƒํ•˜๋ฉด, ์„ค์ •ํ•œ Dockerfile์„ ์‹คํ–‰ํ•˜๋„๋ก Jenkins๋ฅผ ์„ค์ •ํ•œ๋‹ค. ์†Œ์Šค๋ฅผ ๋นŒ๋“œํ•˜๊ณ , ๊ฒฐ๊ณผ๋ฅผ nginx ์ปจํ…Œ์ด๋„ˆ๋ฅผ ์‹คํ–‰ํ•ด ์ ์žฌํ•˜๋„๋ก Dockerfile์„ ์ž‘์„ฑํ•œ๋‹ค. ์†Œ์Šค๋ฅผ ์ˆ˜์ •ํ•˜๊ณ  ์ปค๋ฐ‹ํ•˜๋ฉด, Jenkins๊ฐ€ ์ž๋™์œผ๋กœ ๋นŒ๋“œ/๋ฐฐํฌํ•ด์„œ ๊ฒฐ๊ณผ๋ฅผ ๊ณต์œ ํ•  ์ˆ˜ ์žˆ๋‹ค. ๐ŸŽฏ ์ž‘์—…ํ™˜๊ฒฝ ๊ตฌ์„ฑ 1. WSL2 ์„ค์น˜ ๋งˆ์นจ, ์–ผ๋งˆ์ „ ์ปดํ“จํ„ฐ๋ฅผ ํฌ๋งทํ•ด์„œ WSL์ด ์ง€์›Œ์ง„ ์ƒํƒœ์˜€๋‹ค. ๋งˆ์ดํฌ๋กœ์†Œํ”„ํŠธ์—์„œ ์ œ๊ณตํ•˜๋Š” ์นœ์ ˆํ•œ ๊ฐ€์ด๋“œ๋ฅผ ๋ณด๊ณ  WSL2๋ฅผ์„ ๋‹ค์‹œ ์„ค์น˜ํ–ˆ๋‹ค(1). ๋ฆฌ๋ˆ…์Šค ํ™˜๊ฒฝ์„ ๋” ์พŒ์ ํ•˜๊ฒŒ ์‚ฌ์šฉํ•˜๊ณ ์ž ํ„ฐ๋ฏธ๋„์„ ์„ค์น˜ํ•˜๊ณ (2), ํŽธ์˜์„ฑ์„ ์œ„ํ•ด ๊ด€๋ฆฌ์ž ๊ถŒํ•œ์˜ ๋ฐ”๋กœ๊ฐ€๊ธฐ๋ฅผ ๋ฐ”ํƒ•ํ™”๋ฉด์—..