[python] install matplotlib in python3.x under ubuntu 14.04

sudo apt-get install python3-matplotlib

[python] install numpy under ubuntu 14.04, python3.x

sudo apt-get install python3-numpy

[ubuntu] screenshot area auto save to desktop by shutter ; ask to save but default folder is Desktop by default screeshot app.

1. For original default Ubuntu screenshot app

a.Just install dconf-editor:

sudo apt-get install dconf-editor

b.Open dconf-editor and edit the

auto-save-directory field org > gnome > gnome-screenshot
=>/home/xxxx/Desktop



Ps. shortcut [shift+PrintScrn]  is defaut shortcut for screenshot of area




2. install app shutter

by....

http://apt.ubuntu.com/p/shutter


after configure all setting....


 you need to let original PrintScrn key to call shutter instead the original

screenshot app.


Go to setting -> keyboard -> go down to add custom shortcut
and add new one.

Use the command =>
    Full screen with Shutter – shutter -f
    Take a screenshot of an area with Shutter – shutter -s

[Machine Learning][Study Note][Week1] Matrix

可以利用矩陣運算, 將同一組資料對於不同多個h(x)一次帶入,非常方便

 

一些其他特性

矩陣 I 如果為Identity, A*I=I*A, 則有交換律

否則一般矩陣 A*B != B*A

矩陣有結合律

A+(B+C)=(A+B)+C

可以用軟體求Inverse

A*A(inverse)=I

ex: 3*1/3=1, 則1/3就是 3 的inverse.  note: 0 沒有 inverse.

[Machine Learning][Study Note][Week1] Cost Function & Gradient Descent

一個線性回歸的公式

h(x)=θ0+θ1*x

h(x)=hypothesis



 求 Cost Function 的最小值, 也就是說, 要對 h(x) 找cost 最小的,此h(x) 就會當作

我們的回歸公式

所以, 我們要決定怎樣的東西來當作cost function? 這裡我們用距離的平方,
所以, 找一個一階線性回歸的最佳解, 就是這條線能讓所有訓練資料帶進去
得到所有人距離這條線的總和最小, 那就把係數θ0, θ1拿來用.

Gradient Descent
＝你把普通的兩變數函數畫出來後, X=θ0,Y=θ1,Z=Cost,
你就想像你起始站在山頂上, 但這坐山可能不是最高的山, 然後一步一步的
往山下走, 直到最低點(local min), 所以對於一般的函數來說, 由於
可能有另外一座高山,導致也得到不同的local min2, 因此這這個演算法
並沒有得到全局最min 解. (我們可以控制走得步伐)


所以, 有了cost function 後, 我們要對他求min, 利用Gradient Descent 演算法
來求,

在若在2變數 θ0, θ1 之下, 用軟體畫出所有可能的Y=h(x), 會是一個3D
圖, 而剛好是個凸邊型, 也就是弓型, 而雖然 Gradient Descent 會求出
local min 值, 但是因為這個 cost function 是凸邊型, 所以他的local min
就是全局最小, 所以就得到我們要的答案了.


事實上 梯度下降法能用在所有地方, 不只是線性回歸.
後面還有正規方程 normal equation 也能解最小值.

[Machine Learning][Study Note][Week1] Supervised / Unsupervised Learning

Supervised Learning
=由一些已經知道解答的數據群中, 訓練出模型的方式

這主要分兩大類 1.回歸 2.分群

1.回歸=用於連續值得輸出預測

譬如你有一堆這個區域房子成交價
X= 坪數
Y=售價

你用這些歷史數據訓練出一個回歸模型, 於是你可以利用這個回歸模型去預測新的X

譬如你朋友想要在這區域賣房子, 於是他給你他房子的坪數, 你套入那個練出來的模型

得到Y, 則可以建議他以Y價格售出



2. 分群=用於離散值得輸出預測

譬如

有一堆有腫瘤病患的資料, 而假設我們X只看一個feature 叫做腫瘤直徑
(實際上會有很多feature)

而 Y就是 0=不是惡性腫瘤 1=是惡性腫瘤

所以把歷史資料都畫出來之後, 訓練出一個演算法可以分群, 以便之後出入新的病人的
腫瘤直徑, 可以預測是否為惡性


又

X=腫瘤直徑 , Y=年齡

一樣可以訓練出分群模型








Unsupervised Learning
=這是說你不知道這推數據姿料理有怎樣的規則, 你想要用演算法看看能不能找出這些資料中存在某種結構(可能這些data能分成不同群)

譬如說
每天看到的新聞網頁, 他們去其他網路上收集新聞之後, 他會被自動分到同一類, 譬如某些新聞會被分成運動, 財金 , 這就是跑了演算法之後, 發現能被分類


又或者你有一大堆的健身客戶資料, 你不知道客戶之間有甚關係, 去跑之後, 發現
這些客戶之中, 又有很多膝蓋受過傷的細分市場客戶

簡單來說,
我手上有一堆數據, 我根本不知道裡面有甚東西, 我沒給演算法答案,
請問你（演算法）你可以幫我找出這些數據當中的類型嗎？
我只告訴你演算法, 沒告訴你這些資料的答案(譬如這個人該分到哪群)

[python] debug log with logging config, pprint with logging, logging to file and still output at stdout

import pprint
import logging
from source.projectparameter import project_parameter as pp


def logconfig():
    print_format="%(asctime)s,line:(%(lineno)d): %(message)s"

    logging.basicConfig(filename="abspath/xxxx_debug.log",
                        filemode='a',
                        format=print_format,
                        datefmt='%H:%M:%S',
                        level=logging.DEBUG)
    stderrLogger=logging.StreamHandler()
    stderrLogger.setFormatter(logging.Formatter(print_format))
    logging.getLogger().addHandler(stderrLogger)
    return 0



def show_all_init_paramater():

    for tag in pp:
        #print('\n',tag,'with',len(pp[tag]),'items')
        #logging.debug("\n",tag,"with",len(pp[tag]),"items")
        logging.debug("%s with %s items",tag,len(pp[tag]))
        logging.debug("\n"+pprint.pformat(pp[tag]))
    return 0


def main():
    logconfig()
    logging.debug('=======NEW SECTION======')
    show_all_init_paramater()


    return

main()

[python] Ubuntu, python 3.x list files under folder , list modify time , create time of files. 列出資料夾下的檔案list, 和修改時間或其他屬性

import os,time



file_list=os.listdir("/home/xxxxxx/Desktop")
#print(file_list)


for file in file_list:
    if '~'  in file:
        file_list.remove(file)

print(file_list)

for file in file_list:
    (mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime) = os.stat(file)
    print (file,"\nlast modified: %s" % time.ctime(mtime),'\nCreate time   ',time.ctime(ctime),'\n')

[python] python 3.x , date itreating from start date to stop(today) date.

from datetime import datetime,timedelta


def daterange(begin, end, delta = timedelta(1)):
    """Form a range of dates and iterate over them.

    Arguments:
    begin -- a date (or datetime) object; the beginning of the range.
    end   -- a date (or datetime) object; the end of the range.
    delta -- (optional) a timedelta object; how much to step each iteration.
             Default step is 1 day.

    Usage:

    """
    if not isinstance(delta, timedelta):
        delta = timedelta(delta)

    ZERO = timedelta(0)

    if begin < end:
        if delta <= ZERO:
            raise StopIteration
        test = end.__gt__
    else:
        if delta >= ZERO:
            raise StopIteration
        test = end.__lt__

    while test(begin):
        yield begin
        begin += delta


def main():

    start = datetime(2010, 1, 1,19,00,00)
    stop = datetime.today()
    for x in daterange(start,stop):
        # only when today, it need to download after 7:00pm, so force the start date all is 19:00:00
        if(stop > x):
            #print(stop,x)
            year=str(x.year)
            month=str('%02d' % (x.month))
            day=str('%02d' % (x.day))

            download_xxxxx_from_web(year,month,day)

        else:
            print('For ',x,'Need to wait to 7:00')
    return


main()

[ubuntu] enable ssh login

sudo apt-get install openssh-server

[Ubuntu 14.04.1] 解決Chrome/ Firefox字體變成標楷體或者新細明體的問題

[Ubuntu 14.04.1] 解決Chrome/ Firefox字體變成標楷體或者新細明體的問題

fonts-arphic-ukai,fonts-arphic-uming
這兩個套件

移除他即可

# sudo apt-get remove fonts-arphic-ukai && sudo apt-get remove fonts-arphic-uming



***** logout, login again

[ubuntu] 14.04 install 酷音

1.安裝酷音

sudo apt-get install ibus-chewing

2.

logout, login again, go to text entry , and add

Chinese (Chewing)

 

[python] python numpy static , sum, mean, std (axis = dimension)

a = np.array([[1, 2], [3, 4],[5, 6]])

print('-------------------')
print(np.sum(a))
print(a.sum())
print(a.sum(0))
print(np.sum(a,axis=0))
print(np.sum(a,axis=1))
print('-------------------')
print(np.mean(a))
print(a.mean())
print(a.mean(0))
print(np.mean(a, axis=0))
print(np.mean(a, axis=1))

print('-------------------')
print(np.std(a))
print(a.std())
print(a.std(1))
print(np.std(a, axis=0))
print(np.std(a, axis=1))

output:

-------------------
21
21
[ 9 12]
[ 9 12]
[ 3  7 11]
-------------------
3.5
3.5
[ 3.  4.]
[ 3.  4.]
[ 1.5  3.5  5.5]
-------------------
1.70782512766
1.70782512766
[ 0.5  0.5  0.5]
[ 1.63299316  1.63299316]
[ 0.5  0.5  0.5]
[Finished in 0.1s]

ps: axis 0 means do operator in each column individual. (first dimension.) 

print(np.sum(a,axis=0))

[1,2]

[3,4]

[5,6] 

=[1+3+5 , 2+4+6]= [9,12]

axis 1 means to operator second dimension.  

print(np.sum(a,axis=1)) 

=[1+2,3+4,5+6]=[3,7,11]

---------------------

a = np.array([[1, 3,3, 4, 5, 6],[1, 3,3, 4, 5, 6]])
b = np.array([[1, 2,3, 4, 5, 6],[1, 1,1, 1, 1, 1]])
print(a)
print(np.mean(b, axis=0))
print(a-np.mean(b, axis=0))

[[1 3 3 4 5 6]
 [1 3 3 4 5 6]]

[ 1.   1.5  2.   2.5  3.   3.5]

[[ 0.   1.5  1.   1.5  2.   2.5]
 [ 0.   1.5  1.   1.5  2.   2.5]]
[Finished in 0.1s 

[python] python 3.4 Ubuntu 14.04 install scikit-learn by building from source code.

0.Optional, If you needed....

sudo apt-get install build-essential python3-dev python3-setuptools \
                     python3-numpy python3-scipy \
                     libatlas-dev libatlas3gf-base

1. download scikit-learn source code form

https://pypi.python.org/pypi/scikit-learn/0.15.2#downloads


The same as below, download the first one, scikit-learn-0.15.2.tar.gz

and extract it in your downloaded folder.



2. go to the extracted folder , and build it.

xxxxx/Downloads/scikit-learn-0.15.2$ python3.4 setup.py install --user

 








**if you get the error


 x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-fstack-protector-strong’

 (Solution A:)
edit the below file by sudo

/usr/lib/python3.4/plat-x86_64-linux-gnu/_sysconfigdata_nd.py 

=>this file is belone libpython3.4-minimal package, if you don't have it,
install that.



and replace "all"  -fstack-protector-strong  to -fstack-protector

and saved.


and use the build command above again....

$  python3.4 setup.py install --user





 (Solution B:)
upgrade gcc to 4.9 (I don't want to do that.....)

[IDE] Sublime text 3 , editing config for myself. Tabs to spaces, trim white space when save, draw white sapce for notice.

1. go to  submits -> preferences -> settings - user

copy and pasted below config.


{
    "draw_white_space": "all",
    "font_size": 14,
    "translate_tabs_to_spaces": true,
    "trim_trailing_white_space_on_save": true,
    "auto_match_enabled": false,
    "use_tab_stops": false
}



2.  TrailingSpaces-master package.
Optional: highlight the white space with red color.
 (But in fact, space will be trim when you save file as above setting.)

Go to github to download the zip file for whole project on the right buttom

https://github.com/SublimeText/TrailingSpaces

then extract that and you can see the folder  TrailingSpaces-master.

Go Submits =>Preferences => Browser Packages

Then move the folder TrailingSpaces-master in to the folder which Submits jumped out.

Which means you should pasted TrailingSpaces-master folder in to

sublimes-text-3/Packages/

and place with the /sublimes-text-3/Packages/User folder together.


It will seem like....


xxxx:~/.config/sublime-text-3/Packages$ pwd
/home/xxxx/.config/sublime-text-3/Packages

xxxx:~/.config/sublime-text-3/Packages$ ll
total 24
drwx------  6 xxxx xxxx 4096  3月 22 14:43 ./
drwx------  7 xxxx xxxx 4096  3月 15 17:29 ../
drwxrwxr-x 11 xxxx xxxx 4096  3月 22 14:43 bz2/
drwxr-xr-x  7 xxxx xxxx 4096  3月 15 17:32 ssl-linux/
drwxrwxr-x  3 xxxx xxxx 4096 10月 27 22:54 TrailingSpaces-master/
drwx------  3 xxxx xxxx 4096  3月 29 09:52 User/



You are good to go.