how to install chinese and japanese fonts for matplotlib and seaborn plots

### Jupyter Notebook Python Code for Displaying CJK Unicode
import matplotlib
from matplotlib.font_manager import FontProperties

### 下載日中字型檔
### install japanese font
!apt-get -y install fonts-ipafont-gothic
font_jp = FontProperties(fname=r'/usr/share/fonts/opentype/ipafont-gothic/ipagp.ttf',size=20)
print(font_jp.get_family())
print(font_jp.get_name())

### install chinese font
!apt-get -y install fonts-moe-standard-kai 
font_tw = FontProperties(fname=r'/usr/share/fonts/truetype/moe/MoeStandardKai.ttf',size=20)
print(font_tw.get_family())
print(font_tw.get_name())

### install chinese,japanese,korean font
#!apt-get install fonts-noto-cjk  ## .ttc
#font_cjk = FontProperties(fname=r'/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc',size=20)
#print(font_cjk.get_family())
#print(font_cjk.get_name())

!apt-get install ttf-unifont
font_uni = FontProperties(fname=r'/usr/share/fonts/truetype/unifont/unifont.ttf',size=20)
print(font_uni.get_family())
print(font_uni.get_name())

### 設定畫圖啟用 sans-serif 系列字型
!grep font.family /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc
!sed -i "s/#font.family/font.family/" /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc
!grep font.family /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc

### 設定屬於 sans-serif 系列字型包含日中字型
!grep font.sans-serif /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc
!sed -i "s/#font.sans-serif.*DejaVu Sans/font.sans-serif     : IPAPGothic, TW-MOE-Std-Kai, Unifont, DejaVu Sans/" /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc
#!sed -i "s/font.sans-serif.*DejaVu Sans/font.sans-serif     : IPAPGothic, TW-MOE-Std-Kai, Unifont, DejaVu Sans/" /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc
!grep font.sans-serif /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/matplotlibrc

### 連結日中字型檔到畫圖字型目錄
!ln -s /usr/share/fonts/opentype/ipafont-gothic/ipagp.ttf /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/
!ln -s /usr/share/fonts/truetype/moe/MoeStandardKai.ttf /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/
#!ln -s /usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/
!ln -s /usr/share/fonts/truetype/unifont/unifont.ttf /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/
!ls /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/[Miu]*

### 重建畫圖字型快取，納入日中字型檔
matplotlib.font_manager._rebuild()
flist = matplotlib.font_manager.get_fontconfig_fonts()
names = [matplotlib.font_manager.FontProperties(fname=fname).get_name() for fname in flist]
print(names)

### 確認sans-serif系列清單有日中字型
print(matplotlib.rcParams['font.sans-serif'])  ## 確認sans-serif系列清單是否有日中字型
if 'IPAPGothic' not in matplotlib.rcParams['font.sans-serif']:
  matplotlib.rcParams['font.sans-serif'] = ['IPAPGothic', 'TW-MOE-Std-Kai', 'Unifont'] + matplotlib.rcParams['font.sans-serif']
print(matplotlib.rcParams['font.sans-serif'])

### 測試畫圖字型顯示
#!!!!! 若X軸標示字型有誤，表示引擎快取仍未更新，請【重新啟動並運行所有單元格】
import matplotlib.pyplot as plt

testString = u"喜欢 海灘 散步 걷기 好き"   ## 簡中,繁中,日文,韓文,日文
plt.title(testString, fontproperties=font_uni)
plt.xlabel(testString)   # 利用sans-serif第一個字型顯示
plt.ylabel(testString, fontproperties=font_tw)
plt.show()

import seaborn as sns
emotion_counter = [('愉快', 200), ('高興', 180), ('開心', 160), ('歡喜', 140), ('生氣', 130), ('憤怒', 120), ('悲傷', 110), ('難過', 100), ('哀愁', 90), ('傷感', 80)]
sns.set_color_codes("pastel")
sns.barplot(x=[k for k, _ in emotion_counter], y=[v for _, v in emotion_counter])

參考: 解決Python 3 Matplotlib與Seaborn視覺化套件中文顯示問題 link

Virtual Machine Spec for Google Colab Environment

Google Colab提供最多12小時連線的Jupyter Notebook開發環境，其2018年底層的虛擬機實測規格如下:

CPU: Intel(R) Xeon(R) TwinCore @ 2.20GHz x 2

Memory: 13GB

Drive: 347GB

GPU: Tesla K80 with 4992 cores at 556MHz + 11GB Memory

OS: Ubuntu 18.04.1 LTS

Time Limit: 12 hours

規格實測的python指令如下:

# https://stackoverflow.com/questions/48750199/google-colaboratory-misleading-information-about-its-gpu-only-5-ram-available
# memory footprint support libraries/code
!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi
!pip install gputil
!pip install psutil
!pip install humanize
import psutil
import humanize
import os
import GPUtil as GPU
GPUs = GPU.getGPUs()
# XXX: only one GPU on Colab and isn? guaranteed
gpu = GPUs[0]
def printm():
 process = psutil.Process(os.getpid())
 print("Gen RAM Free: " + humanize.naturalsize( psutil.virtual_memory().available ), " | Proc size: " + humanize.naturalsize( process.memory_info().rss))
 print("GPU RAM Free: {0:.0f}MB | Used: {1:.0f}MB | Util {2:3.0f}% | Total {3:.0f}MB".format(gpu.memoryFree, gpu.memoryUsed, gpu.memoryUtil*100, gpu.memoryTotal))
printm()

!df
!cat /etc/issue
!nvidia-smi
!nvidia-smi -L

!cat /proc/cpuinfo
!cat /proc/meminfo

Use a jupyter and docker environment for learning tensorflow

想要學習Google的開源機器學習套件tensorflow，可以研讀網路上很多Python語言寫的程式範例。
但是，要讓自己的機器能跑得動這些Python寫的tensorflow範例，首先須跨越2個門檻:

1.安裝Python: 
    Python語法分Python2及Python3，IDE介面分jupyter及spyder等，套件管理分anaconda，pip等，
    在選擇眾多之下判斷及安裝皆費時。

2.安裝tensorflow: 
    除了依硬體分成CPU,GPU兩版，軟體也依Windows, Ubuntu, MacOS三種不同作業系統，而分成
    virtualenv、pip、anaconda、docker、原始碼等多種安裝法，各有不同安裝步驟及參數要設定。
    尤其安裝過程到底是使用 Python2 或 Pyhton3 也很容易因此而發生相容性問題。

以下介紹一個利用docker容器原理，快速建立執行tensoflow環境的作法:

1.安裝 docker 容器管理軟體:
    https://www.docker.com/community-edition#/download
       分成 Windows, Linux, Mac等免費版本，下載安裝合適公用版(community edition)
    其中，Windows 10可直接安裝 Docker CE for Windows，內含Windows及Linux兩種核心引擎
    若 Windows 非第10版，建議在Linux虛擬機環境下使用Docker，依序安裝如下軟體:
         VirtualBox Binaries Windows hosts + Ubuntu Desktop + Docker CE for Ubuntu

2.下載 jupyter/tensorflow-notebook 容器映像檔:
    https://hub.docker.com/r/jupyter/tensorflow-notebook/
    利用如下 docker 指令下載容器映像檔，內含Scipy Notebook + Tensorflow and Keras for Python 3.x (without GPU support)
    # docker pull jupyter/tensorflow-notebook
    註: 本映像檔可由如下github提供的 Dockerfile 從Ubuntu製成
         https://github.com/jupyter/docker-stacks/tree/master/tensorflow-notebook 
         https://github.com/jupyter/docker-stacks/tree/master/scipy-notebook
         https://github.com/jupyter/docker-stacks/tree/master/minimal-notebook
         https://github.com/jupyter/docker-stacks/tree/master/base-notebook
         Ubuntu 16.04 (xenial)

3.下載 nfmcclure/tensorflow_cookbook 程式範例:
    https://github.com/nfmcclure/tensorflow_cookbook
    利用如下 git 指令下載範例
    > git clone https://github.com/nfmcclure/tensorflow_cookbook.git
    或
    點選[Download ZIP]下載範例壓縮檔解開

    註: TensorFlow Machine Learning Cookbook範例採用 Pyhton3 撰寫

4.啟動執行範例:
    假設程式範例位於/home/user/docker/tensorflow_cookbook，網頁連線埠為8080，
    則可利用如下指令啟動容器:
    # docker run -it --rm \
        -p 8080:8888 \
        -v /home/user/docker/tensorflow_cookbook:/work \
        --name tflow jupyter/tensorflow-notebook \
        start-notebook.sh --NotebookApp.token='demo' --NotebookApp.notebook_dir='/work'

    然後開啟瀏覽器，利用網址 http://localhost:8080，密碼 demo，連到容器，執行tensorflow範例。

    註: docker run imagePath 指令可將映像檔imagePath放入容器執行，猶如虛擬機開機一般，其參數說明如下:
        -it 保留終端機供互動用，按Ctrl-C可結束容器
        --rm 容器結束時，容器也自動刪除
        -p 8080:8888 將容器內部的8888埠對應到主機的8080埠，供外部使用
        -v outerPath:innerPath 將主機的outerPath目錄掛載到容器內部的innerPath目錄，方便內外硬碟資料共享
        --name tflow 為容器命名為tflow，方便之後取用
        jupyter/tensorflow-notebook 為映像檔在docker hub的標準名稱
        start-notebook.sh  為容器啟用後執行的批次檔，主要啟動jupyter notebook，允許預設8888埠網頁連線
        --NotebookApp.token='demo' 設定網頁連線密碼 demo，以防外人取用
        --NotebookApp.notebook_dir='/work' 設定notebook初始開檔目錄，設為/work可以對應到掛載進來的外部範例目錄，方便取用

   註: 利用docker容器啟動包含jupyter notebook及tensorflow環境的容器，執行tensorflow範例程式的長相如下: