how to curve the exam score by the power function

When a class has poor percentage grades, "take the square root and multiply by 10" is a common technique for raising the grades of the entire class. For finer control of the raising curve, an extension from the square root function to the power/exponential function with an extra RATIO parameter is discussed here. The RATIO parameter can take a recommended value between 0 and 1. It can be roughly seen as the proportion by which the original passing threshold of 60 points is reduced. 1 means that the passing threshold is not reduced, and 0.5 means that it is roughly reduced to 60*0.5=30 points to pass.

Assume the original grade is Old_Score in the range of [0,100] points and the adjusted grade is New_Score in the range of [0,100] points as well. Then using the parameter Ratio in the range of [0,1], the following EXCEL formula for exam score adjustment can be defined:
  

New_Score = ROUNDUP(POWER(Old_Score,Ratio)*100/POWER(100,Ratio),0) 

where ROUNDUP(X,Y) will round X up to the Y decimal places and POWER(X,Y) will compute the result of X raised to the power of Y. Both are EXCEL built-in functions.

Some comments about the above exam score curving formula can be made:

1. When Ratio is equal to 1,  New_Score is equal to Old_Score which means no adjustment.
2. When Ratio is equal to 0.5, New_Score is equal to the square root of Old_Score multiplied by 10 which means the traditional square root curving.
3. When Ratio is equal to 0, New_Score is equal to 100 for all Old_Score except 0 which means the extreme adjustment.
4. For all Ratio in the range of [0,1], this formula will only give non-decreasing adjustment. It means that all adjusted grades will be greater than or equal to the original grades and never exceed the full score of 100 points.

Therefore by properly using Ratio from high (1) to low (0), a spectrum of curving from no adjustment to extreme upward adjustment can be fine tuned. A graph of the curves for different Ratios is shown below. The traditional square root curve is labeled with adjusted scores.

A table for the curving formula is listed below. Each column represents a curving result for a specific Ratio from 0.1 to 1.1 in steps of 0.1. Note that grades in pink color are below 60, the usual passing threshold. Also, when the ratio is 1.1, some scores will decrease, resulting in the undesirable effect of passing grades becoming failing grades.

Ratio	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1	1.1
0	0	0	0	0	0	0	0	0	0	0	0
1	64	40	26	16	10	7	4	3	2	1	1
2	68	46	31	21	15	10	7	5	3	2	2
3	71	50	35	25	18	13	9	7	5	3	3
4	73	53	39	28	20	15	11	8	6	4	3
5	75	55	41	31	23	17	13	10	7	5	4
6	76	57	43	33	25	19	14	11	8	6	5
7	77	59	46	35	27	21	16	12	10	7	6
8	78	61	47	37	29	22	18	14	11	8	7
9	79	62	49	39	30	24	19	15	12	9	8
10	80	64	51	40	32	26	20	16	13	10	8
11	81	65	52	42	34	27	22	18	14	11	9
12	81	66	53	43	35	29	23	19	15	12	10
13	82	67	55	45	37	30	24	20	16	13	11
14	83	68	56	46	38	31	26	21	18	14	12
15	83	69	57	47	39	33	27	22	19	15	13
16	84	70	58	49	40	34	28	24	20	16	14
17	84	71	59	50	42	35	29	25	21	17	15
18	85	71	60	51	43	36	31	26	22	18	16
19	85	72	61	52	44	37	32	27	23	19	17
20	86	73	62	53	45	39	33	28	24	20	18
21	86	74	63	54	46	40	34	29	25	21	18
22	86	74	64	55	47	41	35	30	26	22	19
23	87	75	65	56	48	42	36	31	27	23	20
24	87	76	66	57	49	43	37	32	28	24	21
25	88	76	66	58	50	44	38	33	29	25	22
26	88	77	67	59	51	45	39	35	30	26	23
27	88	77	68	60	52	46	40	36	31	27	24
28	89	78	69	61	53	47	42	37	32	28	25
29	89	79	69	61	54	48	43	38	33	29	26
30	89	79	70	62	55	49	44	39	34	30	27
31	89	80	71	63	56	50	45	40	35	31	28
32	90	80	72	64	57	51	46	41	36	32	29
33	90	81	72	65	58	52	47	42	37	33	30
34	90	81	73	65	59	53	47	43	38	34	31
35	91	82	73	66	60	54	48	44	39	35	32
36	91	82	74	67	60	55	49	45	40	36	33
37	91	82	75	68	61	56	50	46	41	37	34
38	91	83	75	68	62	56	51	47	42	38	35
39	92	83	76	69	63	57	52	48	43	39	36
40	92	84	76	70	64	58	53	49	44	40	37
41	92	84	77	71	65	59	54	50	45	41	38
42	92	85	78	71	65	60	55	50	46	42	39
43	92	85	78	72	66	61	56	51	47	43	40
44	93	85	79	73	67	62	57	52	48	44	41
45	93	86	79	73	68	62	58	53	49	45	42
46	93	86	80	74	68	63	59	54	50	46	43
47	93	86	80	74	69	64	59	55	51	47	44
48	93	87	81	75	70	65	60	56	52	48	45
49	94	87	81	76	70	66	61	57	53	49	46
50	94	88	82	76	71	66	62	58	54	50	47
51	94	88	82	77	72	67	63	59	55	51	48
52	94	88	83	77	73	68	64	60	56	52	49
53	94	89	83	78	73	69	65	61	57	53	50
54	95	89	84	79	74	70	65	62	58	54	51
55	95	89	84	79	75	70	66	62	59	55	52
56	95	90	85	80	75	71	67	63	60	56	53
57	95	90	85	80	76	72	68	64	61	57	54
58	95	90	85	81	77	73	69	65	62	58	55
59	95	90	86	81	77	73	70	66	63	59	56
60	96	91	86	82	78	74	70	67	64	60	58
61	96	91	87	83	79	75	71	68	65	61	59
62	96	91	87	83	79	76	72	69	66	62	60
63	96	92	88	84	80	76	73	70	66	63	61
64	96	92	88	84	80	77	74	70	67	64	62
65	96	92	88	85	81	78	74	71	68	65	63
66	96	93	89	85	82	78	75	72	69	66	64
67	97	93	89	86	82	79	76	73	70	67	65
68	97	93	90	86	83	80	77	74	71	68	66
69	97	93	90	87	84	81	78	75	72	69	67
70	97	94	90	87	84	81	78	76	73	70	68
71	97	94	91	88	85	82	79	77	74	71	69
72	97	94	91	88	85	83	80	77	75	72	70
73	97	94	91	89	86	83	81	78	76	73	71
74	98	95	92	89	87	84	81	79	77	74	72
75	98	95	92	90	87	85	82	80	78	75	73
76	98	95	93	90	88	85	83	81	79	76	74
77	98	95	93	91	88	86	84	82	80	77	76
78	98	96	93	91	89	87	85	82	80	78	77
79	98	96	94	92	89	87	85	83	81	79	78
80	98	96	94	92	90	88	86	84	82	80	79
81	98	96	94	92	90	89	87	85	83	81	80
82	99	97	95	93	91	89	88	86	84	82	81
83	99	97	95	93	92	90	88	87	85	83	82
84	99	97	95	94	92	91	89	87	86	84	83
85	99	97	96	94	93	91	90	88	87	85	84
86	99	98	96	95	93	92	90	89	88	86	85
87	99	98	96	95	94	92	91	90	89	87	86
88	99	98	97	96	94	93	92	91	90	88	87
89	99	98	97	96	95	94	93	92	91	89	88
90	99	98	97	96	95	94	93	92	91	90	90
91	100	99	98	97	96	95	94	93	92	91	91
92	100	99	98	97	96	96	95	94	93	92	92
93	100	99	98	98	97	96	96	95	94	93	93
94	100	99	99	98	97	97	96	96	95	94	94
95	100	99	99	98	98	97	97	96	96	95	95
96	100	100	99	99	98	98	98	97	97	96	96
97	100	100	100	99	99	99	98	98	98	97	97
98	100	100	100	100	99	99	99	99	99	98	98
99	100	100	100	100	100	100	100	100	100	99	99
100	100	100	100	100	100	100	100	100	100	100	100

To see the distribution of boost, a graph and a table showing the score boost (or difference) before and after the adjustment are given below. It can be seen that as Ratio increases, the maximum score boost decreases and occurs at greater original scores.  For example, at Ratio=0.3, a maximum score boost of 42 occurs in the interval of Old_Score=[13,24]. At Ratio=0.5, a maximum score boost of 25 occurs in the interval of Old_Score=[17,35]. And at Ratio=0.7, a maximum score boost of 14 occurs in the interval of Old_Score=[28,33].

In the following table, each column represents the score boost  for a specific Ratio. Maximum score boost in each column is marked in pink color.

Ratio	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1	1.1
0	0	0	0	0	0	0	0	0	0	0	0
1	63	39	25	15	9	6	3	2	1	0	0
2	66	44	29	19	13	8	5	3	1	0	0
3	68	47	32	22	15	10	6	4	2	0	0
4	69	49	35	24	16	11	7	4	2	0	-1
5	70	50	36	26	18	12	8	5	2	0	-1
6	70	51	37	27	19	13	8	5	2	0	-1
7	70	52	39	28	20	14	9	5	3	0	-1
8	70	53	39	29	21	14	10	6	3	0	-1
9	70	53	40	30	21	15	10	6	3	0	-1
10	70	54	41	30	22	16	10	6	3	0	-2
11	70	54	41	31	23	16	11	7	3	0	-2
12	69	54	41	31	23	17	11	7	3	0	-2
13	69	54	42	32	24	17	11	7	3	0	-2
14	69	54	42	32	24	17	12	7	4	0	-2
15	68	54	42	32	24	18	12	7	4	0	-2
16	68	54	42	33	24	18	12	8	4	0	-2
17	67	54	42	33	25	18	12	8	4	0	-2
18	67	53	42	33	25	18	13	8	4	0	-2
19	66	53	42	33	25	18	13	8	4	0	-2
20	66	53	42	33	25	19	13	8	4	0	-2
21	65	53	42	33	25	19	13	8	4	0	-3
22	64	52	42	33	25	19	13	8	4	0	-3
23	64	52	42	33	25	19	13	8	4	0	-3
24	63	52	42	33	25	19	13	8	4	0	-3
25	63	51	41	33	25	19	13	8	4	0	-3
26	62	51	41	33	25	19	13	9	4	0	-3
27	61	50	41	33	25	19	13	9	4	0	-3
28	61	50	41	33	25	19	14	9	4	0	-3
29	60	50	40	32	25	19	14	9	4	0	-3
30	59	49	40	32	25	19	14	9	4	0	-3
31	58	49	40	32	25	19	14	9	4	0	-3
32	58	48	40	32	25	19	14	9	4	0	-3
33	57	48	39	32	25	19	14	9	4	0	-3
34	56	47	39	31	25	19	13	9	4	0	-3
35	56	47	38	31	25	19	13	9	4	0	-3
36	55	46	38	31	24	19	13	9	4	0	-3
37	54	45	38	31	24	19	13	9	4	0	-3
38	53	45	37	30	24	18	13	9	4	0	-3
39	53	44	37	30	24	18	13	9	4	0	-3
40	52	44	36	30	24	18	13	9	4	0	-3
41	51	43	36	30	24	18	13	9	4	0	-3
42	50	43	36	29	23	18	13	8	4	0	-3
43	49	42	35	29	23	18	13	8	4	0	-3
44	49	41	35	29	23	18	13	8	4	0	-3
45	48	41	34	28	23	17	13	8	4	0	-3
46	47	40	34	28	22	17	13	8	4	0	-3
47	46	39	33	27	22	17	12	8	4	0	-3
48	45	39	33	27	22	17	12	8	4	0	-3
49	45	38	32	27	21	17	12	8	4	0	-3
50	44	38	32	26	21	16	12	8	4	0	-3
51	43	37	31	26	21	16	12	8	4	0	-3
52	42	36	31	25	21	16	12	8	4	0	-3
53	41	36	30	25	20	16	12	8	4	0	-3
54	41	35	30	25	20	16	11	8	4	0	-3
55	40	34	29	24	20	15	11	7	4	0	-3
56	39	34	29	24	19	15	11	7	4	0	-3
57	38	33	28	23	19	15	11	7	4	0	-3
58	37	32	27	23	19	15	11	7	4	0	-3
59	36	31	27	22	18	14	11	7	4	0	-3
60	36	31	26	22	18	14	10	7	4	0	-2
61	35	30	26	22	18	14	10	7	4	0	-2
62	34	29	25	21	17	14	10	7	4	0	-2
63	33	29	25	21	17	13	10	7	3	0	-2
64	32	28	24	20	16	13	10	6	3	0	-2
65	31	27	23	20	16	13	9	6	3	0	-2
66	30	27	23	19	16	12	9	6	3	0	-2
67	30	26	22	19	15	12	9	6	3	0	-2
68	29	25	22	18	15	12	9	6	3	0	-2
69	28	24	21	18	15	12	9	6	3	0	-2
70	27	24	20	17	14	11	8	6	3	0	-2
71	26	23	20	17	14	11	8	6	3	0	-2
72	25	22	19	16	13	11	8	5	3	0	-2
73	24	21	18	16	13	10	8	5	3	0	-2
74	24	21	18	15	13	10	7	5	3	0	-2
75	23	20	17	15	12	10	7	5	3	0	-2
76	22	19	17	14	12	9	7	5	3	0	-2
77	21	18	16	14	11	9	7	5	3	0	-1
78	20	18	15	13	11	9	7	4	2	0	-1
79	19	17	15	13	10	8	6	4	2	0	-1
80	18	16	14	12	10	8	6	4	2	0	-1
81	17	15	13	11	9	8	6	4	2	0	-1
82	17	15	13	11	9	7	6	4	2	0	-1
83	16	14	12	10	9	7	5	4	2	0	-1
84	15	13	11	10	8	7	5	3	2	0	-1
85	14	12	11	9	8	6	5	3	2	0	-1
86	13	12	10	9	7	6	4	3	2	0	-1
87	12	11	9	8	7	5	4	3	2	0	-1
88	11	10	9	8	6	5	4	3	2	0	-1
89	10	9	8	7	6	5	4	3	2	0	-1
90	9	8	7	6	5	4	3	2	1	0	0
91	9	8	7	6	5	4	3	2	1	0	0
92	8	7	6	5	4	4	3	2	1	0	0
93	7	6	5	5	4	3	3	2	1	0	0
94	6	5	5	4	3	3	2	2	1	0	0
95	5	4	4	3	3	2	2	1	1	0	0
96	4	4	3	3	2	2	2	1	1	0	0
97	3	3	3	2	2	2	1	1	1	0	0
98	2	2	2	2	1	1	1	1	1	0	0
99	1	1	1	1	1	1	1	1	1	0	0
100	0	0	0	0	0	0	0	0	0	0	0

In conclusion, the traditional square root curving method is a special case of the discussed power/exponential curving method. Compared to the square root curving method, the power/exponential curving method with a single tuning parameter can give you an easier and finer control of the entire class grades to meet your desired statistics (say, the average or the flunk ratio) requirement.

References:
1.Richeson (2008), How to curve an exam and assign grades, root functions
2.Nicolai (2008), How to Curve an Exam

weka tutorial test 2

利用weka機器學習軟體的Explorer/Preprocess,Classify,Select_Attribute功能,
回答如下問題:

離散化(參考weka書17章4.5節)
1.針對糖尿病判定資料集diabetes,如下離散化方法
  何者最有助於J48分類器提升未見資料準確率?
   unsupervised Discretize (makeBinary=false)
   unsupervised Discretize (makeBinary=true)
   supervised Discretize (makeBinary=false)
   supervised Discretize (makeBinary=true)
  提示:使用10次交叉驗證測試準確率
       supervised Discretize要經由FilteredClassifier+J48作測試才公平(不會一魚兩吃)


排除重複屬性能力(參考weka書17章4.11節)
2.針對糖尿病判定資料集diabetes,若挑第1個屬性複製2次,成3個相依屬性,
  則如下三種挑選屬性方法,何者最能排除重複屬性,何者慎選屬性後準確率最高?
   A.InfoGainAttributeEval+Ranker(只留8個屬性)
 B.CfsSubsetEval+BestFirst
 C.WrapperSubsetEval+NaiveBayes+BestFirst
  請利用AttributeSelectedClassifier + NaiveBayes製作下表,回答問題:
      準確率 挑選屬性集合
 A.InfoGainAttributeEval+Ranker
 B.CfsSubsetEval+BestFirst
 C.WrapperSubsetEval+NaiveBayes+BestFirst

參數調整(參考weka書17章4.12節)
3.針對糖尿病判定資料集diabetes,找出最近鄰居法IBk最適用鄰居數k值為何?
 提示:利用weka.classifiers.meta.CVParameterSelection
  變化鄰居數k=1~10分成10步

文字資料集(參考weka書17章5.4節)
4.針對如下新聞資料集找出判定穀物新聞準確率最高的分類器(含不同參數).
 訓練集: ReutersGrain-train.arff
 測試集: ReutersGrain-test.arff
  列出評比過的分類器,含各種參數變化,依準確率高到低排序.
  提示:文章向量化工具StringToWordVector,採預設參數
       StringToWordVector要經由FilteredClassifier作測試才公平(不會一魚兩吃)
       使用提供測試集選項所得之準確率,列出至少3個數據.


參考文獻:
1.weka軟體下載,內含所有資料集,
   http://www.cs.waikato.ac.nz/~ml/weka/index_downloading.html

2.witten-11-mkp-data mining- practical machine learning tools and techniques with java implementations
  第17章tutorial exercises for the weka explorer

  Explorer::Classify
  離散化
 4.1.glass by unsupervised Discretize
   weka.filters.unsupervised.attribute.Discretize
     equal-width (預設值) v.s. equal-frequency
   觀察等區間寬度離散化結果各區間案例分佈圖
   等案例數離散化結果某些屬性區間案例分佈仍很偏斜,理由為何?
   哪些離散化屬性適合作預測

 4.2-4.3.glass by supervised Discretize
   weka.filters.supervised.attribute.Discretize
     類別分佈保持一致性?
   --
   某些屬性仍唯持一個區間,不作切割,理由為何?

 4.4.glass by supervised/unsupervised Discretize
   任挑一離散化過濾器,啟用建立兩值屬性makeBinary功能,觀察結果
   產生的兩值屬性意義為何?

 4.5.ionosphere by unsupervised Discretize + J48
   填入下表:
                交叉驗證準確率, 樹節點數
   (1)原始資料
   (2)非監督式離散化
      (makeBinary=false)
   (3)非監督式離散化
      (makeBinary=true)

 4.6.-4.7.ionosphere by FilteredClassifier
   過濾器:supervised Discretize + 分類器:J48
                交叉驗證準確率, 樹節點數
   (4)監督式離散化
      (makeBinary=false)
   (5)監督式離散化
      (makeBinary=true)
   --
   為何決策樹使用離散資料,預測表現比原始資料還好?

  Explorer::Attribute Selection
 filter:
   weka.attributeSelection.CfsSubsetEval
     weka.attributeSelection.BestFirst
   weka.attributeSelection.InfoGainAttributeEval
     weka.attributeSelection.Ranker
 wrapper:
   weka.attributeSelection.WrapperSubsetEval
   weka.attributeSelection.AttributeSelectedClasifier

 4.8.labor by InfoGainAttributeEval + Ranker
   利用information gain找出labor資料集4個最重要屬性
       挑選屬性集合
   InfoGainAttributeEval+Ranker

 4.9.labor by CfsSubsetEval+BestFirst / WrapperSubsetEval+J48+BestFirst
     挑選屬性集合
   CfsSubsetEval+BestFirst
   WrapperSubsetEval+J48+BestFirst
   -
   哪些屬性兩個方法都挑選到?
   兩法共挑屬性和InfoGainAttributeEval+Ranker所挑屬性有何關係.

 4.10.diabetes by NaiveBayes
   NaiveBayes啟用useSupervisedDiscretization=true
   第1屬性拷貝次數 0,1,2,3,4
   準確率

 4.11.diabetes by AttributeSelectedClassifier
   分類器: NaiveBayes
   挑選屬性法:
     InfoGainAttributeEval+Ranker(只留8個屬性)
     CfsSubsetEval+BestFirst
     WrapperSubsetEval+NaiveBayes+BestFirst
   觀察三種屬性挑選法在屬性重複多次下的表現
   可否成功剔除重複屬性,若不行,理由為何?
      準確率 挑選屬性
     InfoGainAttributeEval+Ranker
     CfsSubsetEval+BestFirst
     WrapperSubsetEval+NaiveBayes+BestFirst

  參數調整
   weka.classifiers.meta.CVParameterSelection

   4.12.diabetes by CVParameterSelection+IBk
     IBk變化鄰居數K=1,10,10步
       交叉驗證準確率
     IBk(k=1)
     IBk(k=2)
     IBk(k=3)
     IBk(k=4)
     IBk(k=5)
     IBk(k=6)
     IBk(k=7)
     IBk(k=8)
     IBk(k=9)
     IBk(k=10)
     --
     所挑k值為何?

   4.13.diabetes by CVParameterSelection+J48
     葉節點最小案例數M=1,10,10步
     修剪信心度C=0.1,0.5,5步
     準確率有變嗎,樹節點數為何,所挑M,C為何?

  文件分類
 weka.filters.unsupervised.attribute.StringToWordVector
 mini-train訓練文件:
   class text
   yes the price of crude oil has increased significantly
   yes demand for crude oil outstrips supply
   no some people do not like the flavor of olive oil
   no the food was very oily
   yes crude oil is in short supply
   no use a bit of cooking oil in the frying pan
 mini-test測試文件:
   class text
   ? oil platforms extract crude oil
   ? canola oil is supposed to be healthy
   ? iraq has significant oil reserves
   ? there are different types of cooking oil

 5.1.-5.3.mini-xx by FilteredClassifier
    過濾器:StringToWordVector + 分類器:J48
    mini-train + mini-test
   針對訓練文件mini-train
   StringToWordVector採預設選項,產生多少屬性?
   改minTermFreq選項為2,產生多少屬性?
   --
   利用minTermFreq=2產生資料,建立J48決策樹
   --
   利用前述決策樹,預測mini-test文件結果

  真實文件
 5.4.reutersxx_yy.arff by J48 and NaiveBayesMultinominal
  ReutersCorn-train.arff + ReutersCorn-test.arff
  ReutersGrain-train.arff + ReutersGrain-test.arff
  weka.classifiers.meta.FilteredClassifier
   填入下表:
   預測準確率  J48 NaiveBayesMultinominal
   corn-train+corn-test
   grain-train+grain-test
   -
   哪一個分類器表現較好?

weka tutorial test 1

利用weka機器學習軟體的Explorer/Classify功能,
針對刑事鑑定玻璃種類glass資料集,
嘗試各種分類器,含各種參數變化,
回答如下問題:

1.那一種分類器最能遷就既有資料,可就看過資料表現最好?
  列出評比過的分類器,含各種參數變化,依準確率高到低排序.
 提示:參考weka書17章1.9節,使用訓練集測試準確率

2.那一種分類器最能預測將來資料,可就未見資料表現最好?
  列出評比過的分類器,含各種參數變化,依準確率高到低排序.
 提示:參考weka書17章2.1-2.3節,使用10次交叉驗證測試準確率

3.就第2題所列前3名分類器中,何者耐雜訊能力最強,受類別雜訊干擾最少?
  列出前3名分類器,依雜訊量由0%升到50%,準確率降低比例,由低到高排序.
 提示:參考weka書17章2.6-2.8節,添加類別雜訊以交叉驗證作準確率測試

4.就第2題所列前3名分類器中,何者受訓練集大小之影響最小?
  列出前3名分類器,依訓練集量由100%降到50%,準確率降低比例,由低到高排序.
 提示:參考weka書17章2.9-2.11節,添加類別雜訊以交叉驗證作準確率測試

註:準確率降低比例的計算公式為:
   若參數變化前準確率p_old,參數變化後準確率p_new,
   則準確率降低比例=(p_old - p_new) / p_old

參考文獻:
1.weka軟體下載,內含glass資料集,
   http://www.cs.waikato.ac.nz/~ml/weka/index_downloading.html

2.witten-11-mkp-data mining- practical machine learning tools and techniques with java implementations
  第17章tutorial exercises for the weka explorer

  Explorer::Classify
 1.8.weather.nominal by J48
   weka.classifiers.trees.J48
   利用J48 -C.025 -M 2學習後,觀看決策樹,
     sunny,cool,high,true案例應歸屬何類別

 1.9.iris by J48 -C.025 -M 2,
   訓練集測試準確率
   交叉驗證測試準確率
   何者較可信

 1.10.iris by J48 -C.025 -M 2,
   Visualize classifier errors分析錯誤位置

 2.1.-2.3.glass.arff by IBk
   weka.classifiers.lazy.IBk
   屬性數,屬性名,類別屬性值
   10切片交叉驗證, 單一鄰居數
   --
   IBk(k=1)準確率
   --
   IBk(k=5)準確率

 2.4.-2.5.glass by IBk
   人工挑選屬性子集合
   屬性子集合大小,最佳屬性子集合,分類準確率
  9
  8
  7
  6
  5
  4
  3
  2
  1
  0
   --
   此最佳準確率有無偏頗,是否適用於未見資料?

 2.6.-2.8.glass by IBk
   weka.filters.unsupervised.attribute.AddNoise
   weka.classifiers.meta.FilteredClassifier
     過濾器只套用第1輪學習過程,跳過第2輪測試過程
     讓雜訊只干擾學習集資料,不干擾測試集資料
   -
   不同鄰居數IBk應付類別雜訊效果(交叉驗證準確率)
   雜訊量百分比,k=1,k=3,k=5
   0%
   10%
   20%
   30%
   40%
   50%
   60%
   70%
   80%
   90%
   100%
   --
   類別雜訊量變大,準確率變化效果
   --
   鄰居數變化,準確率變化效果

 2.9.-2.11.glass by IBk (k=1) and J48
   weka.filters.unsupervised.instance.Resample
   -
   IBk(k=1)及J48受訓練集大小影響效果
   訓練集百分比,IBk(k=1),J48
   10%
   20%
   30%
   40%
   50%
   60%
   70%
   80%
   90%
   100%
   --
   訓練集百分比變高之影響?
   --
   IBk或J48誰受訓練集百分比影響較大?

auto mount windows shares on linux filesystem

如何讓unix可以自動掛載windows分享資料夾:

實例: Linux CentOS 5.4 + Windwos XP
目的: 要將Windows主機smbhost所分享出來的share資料夾,
     其存取帳號為user,密碼為pass,
     以唯讀,uid=xxx,gid=yyy身份
     掛載在Linux主機的/mnt/smbhost/share目錄上供存取
方法:
1.root建立linux掛載點
 #mkdir /mnt/smbhost

2.root修改linux自動掛載相關組態檔,credentials帳密另存其他位置
 #vi /etc/auto.master
   /mnt/smbhost /etc/auto.smbhost
 #vi /etc/auto.smbhost
   share  -fstype=cifs,ro,noperm,credentials=/root/smbhost.autofs,uid=xxx,
gid=yyy ://smbhost/share
 #vi /root/smbhost.autofs
   username=user
   password=pass

3.root保護linux自動掛載相關組態檔
 #chmod 600 /etc/auto.smbhost /root/smbhost.autofs

4.重新啟動自動掛載服務autofs
 #/sbin/service autofs restart

5.一般用戶利用cd進到自動掛載目錄取用資源
 > cd /mnt/smbhost/share

6.採用自動掛載前可先用如下samba客戶端指令測試遠方資料夾的帳密及內容正確與否:
 > smbclient -L smbhost -U user    # 以帳密user/pass,列出smbhost主機有何分享資料夾
 Password: 

 > smbclient '//smbhost/share' -U user  # 以帳密user/pass,連上分享資料夾//smbhost/share
 Password: 
 Domain=[SATURN] OS=[Windows 5.1] Server=[Windows 2000 LAN Manager]
 smb: \> help    # 列舉可用指令
 smb: \> ls      # 列舉目錄內容
 smb: \> cd folder  # 切換到foler資料夾
 smb: \foler\> ls

參考:
http://wiki.centos.org/zh-tw/TipsAndTricks/WindowsShares
http://linux.vbird.org/linux_server/0370samba.php

turn old pc to ad hoc desktop connectors

想要善用舊電腦,最佳方法不外是拿來當專用伺服器或客戶端使用.
以下介紹如何利用thinstation軟體,製作專用桌面連接器.
特色如下:
1.提供rdp/vnc/xdm協定的連線客戶端程式,可連接windows/unix桌面.
2.可製成iso映像檔,配合grub2,從硬碟開機(本文採用);
  或直接syslinux安裝到硬碟,從硬碟開機(待嚐試).
3.硬碟只耗掉100MB空間,適用於硬碟容量少之舊型電腦.
4.記憶體只耗掉100MB,適用於記憶體容量少之舊型電腦.
5.提供各種顯示卡及網路卡驅動程式,適用於舊型電腦.

注意:安裝grub2是為了能從硬碟的iso檔啟動開機,
  目前支援grub2的為ubuntu 10.04 LTS以上,
  故首先安裝ubuntu軟體.以下操作皆在ubuntu上進行.

A.安裝ubuntu,供切割硬碟及安裝grub2
  從 http://www.ubuntu.com/download/ubuntu/download
  選擇 ubuntu 10.04 LTS, 32-bit (recommended) 下載iso,約686MB
  燒錄成光碟,從光碟開機進入LiveCD,不需安裝ubuntu到硬碟.

B.製作一塊>=100MB空間的硬碟切割,供安裝grub2及iso用
  從System/Administration/Partition Editor選單,開啟硬碟切割程式.
  利用調大小/搬移(Resize/Move),新增(New),格式化(Format To)操作,
  找出一塊>=100MB空間,格式化成ext3檔案格式,以下假設該空間為 /dev/sda2.
  
C.安裝grub2,供硬碟iso開機用
  從Applications/Accessories/Terminal選單,開啟終端機.
  利用如下指令安裝grub2,
    > sudo -s
    # mount /dev/sda2 /mnt
    # grub-install --no-floppy --root-directory=/mnt /dev/sda
        本指令將在sda硬碟的mbr安裝grub2開機程式,
        並在/mnt/boot/grub/*產生grub2開機所需相關檔案,
         grub2組態檔為/mnt/boot/grub/grub.cfg,稍後再手動補充
    # mkdir /mnt/boot/iso

D.安裝thinstation軟體及製作thinstation.iso
  D1.下載Thinstation-2.2.2g.tar.bz2軟體,約101MB,
    http://sourceforge.net/projects/thinstation/files/thinstation/thinstation-2.2/Thinstation-2.2.2g.tar.bz2/download
  D2.解開軟體
     # tar jxvf Thinstation-2.2.2g.tar.bz2
  D3.修改組態檔1,設定顯示卡,網路卡,聲音卡等驅動程式及套件
    # vi Thinstation-2.2.2g/build.conf
  D4.修改組態檔2,設定桌面連線程式,網路連線組態,儲存體路徑等
    # vi Thinstation-2.2.2g/thinstation.conf.buildtime
        系統可以有很多個螢幕(SCREEN),螢幕相當於桌面,預設SCREEN=0;
        每個螢幕可以有1,2,3,4四個工作區(WORKSPACE),工作區相當於桌面頁籤,預設WORKSPACE=1;
        每支程式跑起來稱作一個連線(SESSION),
        利用SESSION_#_TYPE設定桌面程式圖示供點選時,每個連線編號(#)必需不同.
  D5.製作 .iso
    # ./Thinstation-2.2.2g/build
         建立包含如上開機,桌面,網路所需開機光碟映像檔thinstation.iso 
         此.iso檔案大小取決於Thinstation-2.2.2g/build.conf選用模組多寡.
         正常70~80MB左右.
    # cp Thinstation-2.2.2g/boot-images/iso/thinstation.iso /mnt/boot/iso/
         將開機光碟映像檔thinstation.iso 拷貝到 grub2 開機硬碟切割中.

E.編輯grub2的grub.cfg開機選單,供硬碟啟動thinstation.iso之用
    # vi /mnt/boot/grub/grub.cfg
     set timeout=5
     set default=0
     menuentry "Thinstation-2.2.2g-Customed-LiveCD" {
       loopback loop (hd0,2)/boot/iso/thinstation.iso
       linux (loop)/vmlinuz load_ramdisk=1 ramdisk_blocksize=4096 root=/dev/ram0 ramdisk_size=120000 splash=silent gfxpayload=1024x768 console=ttyS3
       initrd (loop)/initrd
     }
     
     menuentry "ubuntu-9.04-desktop-i386.iso" {
       loopback loop (hd0,2)/boot/iso/ubuntu-9.04-desktop-i386.iso
       set root=(loop)
       linux /casper/vmlinuz boot=casper iso-scan/filename/boot/iso/ubuntu-9.04-desktop-i386.iso
       initrd /casper/initrd.gz
     }

     menuentry "Windows XP" {
       set root=(hd0,1)
       chainloader +1
     }

     menuentry "reboot" {
       reboot
     }

     以上假設第1顆硬碟sda的第1切割sda1即(hd0,1)安裝Windows XP,
     第2切割sda2即(hd0,2)安裝grub2開機系統及ubuntu 9.04,thinstation兩種.iso.

   # reboot
     重新開機之後,就會看到4個選項,如下,



     預設5秒進入第1個選項即thinstation.iso.
     進入thinstation畫面如下,



     每種連線協定皆有兩個圖示,有加(fs)者為全螢幕(fullscreen).
     可點選其中的MS Windows圖示,
     在Server Name or IP Number,輸入欲連線的Windows IP,
     即以rdesktop套件,走rdp協定,連線到Windows桌面.
     若想要全螢幕,可在Options,輸入 -f 參數,事後即可用Ctrl-Alt-Enter作全半螢幕切換.

註1: grub2的grub.cfg選單本來設計成程式自動產生,但這裏只用其開機掛載iso功能,
    故為求精簡,直接手動編輯.
註2: 此>=100MB硬碟切割其實適合安裝於任何開機硬碟中,
    所佔空間少,基本的mount,ssh.scp,e3vi功能也都有,可當作救援開機碟用.
    前提是grub2要安裝佔用開機硬碟的mbr (master boot record)即可.
註3:從左下Start功能表,開啟rxvt終端機,用free指令,可觀察目前所用記憶體.
    正常只用rdesktop應該不超過100MB,
    若你當初在Thinstation-2.2.2g/build.conf選用較多模組,所用記憶體可能到達170MB.

ssh without password

使用ssh協定,要登入遠方主機或拷貝遠方檔案時,有免去輸入帳密方法,如下:
  原理:  將自己的公開金鑰(localhost:~/.ssh/*.pub)加入
        遠方主機的授權金鑰檔內(remotehost:~/.ssh/authorized_keys)

  工具:  可利用以下reg_ssh_key.sh批次檔作登入及加入動作.
        以後,利用終端機ssh remotehost或scp file remotehost:時,就可以免問帳密.

  方法:  localhost> vi reg_ssh_key.sh    # 建立註冊批次檔,user為遠端主機帳號
          cat ~/.ssh/*.pub | ssh user@$* "cat >> .ssh/authorized_keys"

        localhost> sh reg_ssh_key.sh remotehost  # 利用註冊批次檔,向遠方主機remotehost註冊
金鑰
        Password:

        localhost> ssh remotehost   # 測試ssh自動登入遠方主機,不問密碼

        remotehost>

註:
1.遠方主機的檔案目錄必需設定權限如下:
  remotehost:~/.ssh                 目錄權限為711,即己/群/他權限分別為rwx/x/x,
  remotehost:~/.ssh/authorized_keys 檔案權限為644,即己/群/他權限分別為rw/r/r,

2.個人金鑰生成法:
  若有ssh-user-config指令,以問答方式產生個人金鑰,
  若無該指令,可用ssh-keygen指令產生

3.本地/遠方所需ssh組態檔如下,
  客戶端: /etc/ssh_config 或 /etc/ssh/ssh_config
  伺服端: /etc/sshd_config 或 /etc/ssh/sshd_config

------------------  reg_ssh_key.sh ----------------------
#
# http://josephjiang.com/article/understand-ssh-key/
#
#  run to get /etc/ssh_config, /etc/sshd_config
#     ssh-host-config
#  run to get ~/.ssh/identify.pub, id_rsa.pub, id_dsa.pub
#     ssh-user-config
#     #ssh-keygen -t rsa1
#     #ssh-keygen -t rsa
#     #ssh-keygen -t dsa
#  chmod 711 remote:~/.ssh
#  chmod 644 remote:~/.ssh/authorized_keys
#
cat ~/.ssh/*.pub | ssh user@$* "cat >> .ssh/authorized_keys"