技術ノート

https://detail.chiebukuro.yahoo.co.jp/qa/question_detail/q11207037291 https://nico-tube.net/thug-life-glasses/

threading https://ja.stackoverflow.com/questions/60539/subprocess-popenのstdoutとstderrをリアルタイムに取得する # python 3.9 import subprocess from threading import Thread def main(): command = [ 'mycommand', '-opt1', ] proc = subprocess.Popen( command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) def read_stdout(stdout): for line in stdout: line_text = line.decode('utf-8').strip() print(f'STDOUT: {line_text}', flush=True) print('stdout closed') # closed when process exited def read_stderr(stderr): for line in stderr: line_text = line.decode('utf-8').strip() print(f'STDERR: {line_text}', flush=True) print('stderr closed') # closed when process exited Thread(target=read_stdout, args=(proc.stdout,)).start() Thread(target=read_stderr, args=(proc.stderr,)).start() while True: returncode = proc.poll() if returncode is not None: break time.sleep(0.01) print(f'exited {returncode}') asyncio https://stackoverflow.com/questions/28492103/how-to-combine-python-asyncio-with-threads https://qiita.com/matsui-k20xx/items/4d1c00c4eefd60ba635b import time import asyncio from concurrent.futures import ThreadPoolExecutor async def main(): loop = asyncio.get_event_loop() executor = ThreadPoolExecutor(2) def operation(): time.sleep(1) print('aaa', flush=True) # 2個ずつ実行される（asyncio.runの実行終了まで2秒かかる） loop.run_in_executor(executor, operation) loop.run_in_executor(executor, operation) loop.run_in_executor(executor, operation) loop.run_in_executor(executor, operation) # main関数の実行はブロックされない print('before aaa') asyncio.run(main()) import time import asyncio from asyncio.subprocess import create_subprocess_exec, PIPE from concurrent.futures import ThreadPoolExecutor async def main(): command = [ 'mycommand', '-opt1', ] proc = await create_subprocess_exec( command[0], *command[1:], stdout=PIPE, stderr=PIPE, ) loop = asyncio.get_event_loop() executor = ThreadPoolExecutor() def read_stdout(stdout): while True: line = asyncio.run_coroutine_threadsafe(stdout.readline(), loop).result() if not line: break line_text = line.decode('utf-8').strip() print(f'STDOUT: {line_text}', flush=True) print('stdout closed') # closed when process exited def read_stderr(stderr): while True: line = asyncio.run_coroutine_threadsafe(stderr.readline(), loop).result() if not line: break line_text = line.decode('utf-8').strip() print(f'STDERR: {line_text}', flush=True) print('stderr closed') # closed when process exited loop.run_in_executor(executor, read_stdout, proc.stdout) loop.run_in_executor(executor, read_stderr, proc.stderr) await proc.wait() print(f'exited {proc.returncode}') https://stackoverflow.com/questions/45600579/asyncio-event-loop-is-closed-when-getting-loop 終了時にEvent loop is closedというエラーが出ることがある？ ...

https://blog.foresta.me/posts/convert_mp4_to_hls_with_ffmpeg/ https://yosshi.snowdrop.asia/2015/07/27/1156/ https://ffmpeg.org/ffmpeg-formats.html ffmpeg -i video.mp4 -f hls -hls_time 9 -hls_playlist_type vod -hls_segment_filename "stream%3d.ts" stream.m3u8 ffmpeg -i video.mp4 -c:v libx264 -c:a aac -f hls -hls_time 9 -hls_playlist_type vod -hls_segment_filename "stream%3d.ts" stream.m3u8 https://paulownia.hatenablog.com/entry/2020/10/18/163104 https://hub.docker.com/_/nginx default.conf.template server { root /webroot; location ~* \.ts$ { types { video/MP2T ts; } } location ~* \.m3u8$ { types { application/vnd.apple.mpegurl m3u8; # application/x-mpegURL m3u8; } } } niconico: duration: 6s, application/vnd.apple.mpegurl, video/MP2T （durationはキーフレームとかいろいろで勝手に変わるかも？） ffmpeg -i video.mp4 \ -acodec copy \ -vcodec copy \ -vbsf h264_mp4toannexb \ -map 0 \ -f segment \ -segment_format mpegts \ -segment_time 30 \ -segment_list stream.m3u8 \ -segment_list_flags \ -cache stream%03d.ts codecをコピーすると1ファイルのtsに出力されてしまう場合がある libx264で再エンコードする ...

23系で最新23.0.5に更新 docker-compose pull # nextcloud:23 docker-compose exec -u www-data app php occ db:add-missing-indices docker-compose exec -u www-data app php occ maintenance:repair docker-compose exec db mysql_upgrade -p drawioアプリの互換性 https://github.com/pawelrojek/nextcloud-drawio/issues/177 docker-compose pull # nextcloud:24 docker-compose exec -u www-data app php occ db:add-missing-indices docker-compose exec -u www-data app php occ maintenance:repair docker-compose exec db mysql_upgrade -p

Python 3.7以降の場合、標準モジュールimportlib.resourcesが利用できる。 https://docs.python.org/ja/3/library/importlib.html#module-importlib.resources 以下のようなディレクトリ構造でファイルを同梱する。 setup.py MANIFEST.in README.md LICENSE mymodule __init__.py mymodule.py mydirectory __init__.py myfile1.txt myfile2.bin 注意点 読み込むファイルのあるディレクトリに__init__.pyを作成する MANIFEST.inでファイルがパッケージに同梱されるように記述する include README.md include LICENSE recursive-include mymodule/mydirectory * import importlib.resources as ILR # テキストファイル ILR.read_text('mymodule.mydirectory', 'myfile1.txt', encoding='utf-8') # -> str # バイナリファイル ILR.read_binary('mymodule.mydirectory', 'myfile2.bin') # -> bytes

https://gist.github.com/aoirint/c231ab43436ce6c3aafe2097dbb77f0e クローラの時刻情報を一度Local Naiveで保存してしまったのでUTC Awareに変換するために作成した （まあ変換処理事故って何度もデータ飛ばしてしまったんだけど） https://twitter.com/aoirint/status/1524287849759408129

docker-pidパッケージが便利 https://www.glia-computing.com/blog/listing-process-ids-on-docker-containers/ https://github.com/masatanish/docker-pid pip3 install docker==5.0.3 docker-pid==0.0.3

shift+wで設定を保存 https://unix.stackexchange.com/questions/424486/linux-command-top-saving-configuration shift+mでメモリ使用率順にソート fから項目にSWAPを追加 https://beyondjapan.com/blog/2016/02/change-sort-top/ https://qiita.com/k0kubun/items/7368c323d90f24a00c2f eで表部分のメモリ量スケールを変更 shift+eでサマリー部分のメモリ量スケールを変更

https://gist.github.com/aoirint/ddbc49898d39cc0faa15765db8aa247a

https://www.jma.go.jp/bosai/forecast/ エリア一覧 https://www.jma.go.jp/bosai/common/const/area.json https://www.jma.go.jp/bosai/forecast/const/forecast_area.json https://www.jma.go.jp/bosai/forecast/const/en_amedas.json 祝日一覧？ https://www.jma.go.jp/bosai/forecast/const/anniversary.json 週間用エリアID一覧？ https://www.jma.go.jp/bosai/forecast/const/week_area.json 枝番エリア一覧？ https://www.jma.go.jp/bosai/forecast/const/week_area05.json https://www.jma.go.jp/bosai/forecast/const/week_area_name.json 天気予報 日間概況 https://www.jma.go.jp/bosai/forecast/data/overview_forecast/{ID}.json 東京地方ID: 130000 週間概況 https://www.jma.go.jp/bosai/forecast/data/overview_week/{ID}.json 天気予報（3日先まで、7日先まで） https://www.jma.go.jp/bosai/forecast/data/forecast/{ID}.json 日間警報 https://www.jma.go.jp/bosai/warning/data/warning/{ID}.json 定数を調べる: console.log(JSON.stringify(Forecast)) Pythonラッパー https://github.com/aoirint/aoirint_jmapy https://pypi.org/project/aoirint-jmapy/ 整形例 https://gist.github.com/aoirint/52c778ea00a9b1d080299eea2ced67b3