其中我認為這一題可以當作訓練經濟思考的好東西:
假設有一個國家目前是實施對勞動所得課徵單一稅率等於20%的稅制。此一國家從明年開始將實施稅率在10%和30%兩者間輪流的新稅制，意謂2008 的單一稅率是10%，2009 的單一稅率是30%，之後則以此類推。此一新稅制對勞動供給和可課稅的勞動所得會有何影響？
你認為會發生什麼事?
Friday, June 01, 2007
公共期中考題
這學期修了一門公共經濟學. 前幾天都在想問題
其中我認為這一題可以當作訓練經濟思考的好東西:
你認為會發生什麼事?
其中我認為這一題可以當作訓練經濟思考的好東西:
你認為會發生什麼事?
Dear Dr. Yu
今天收到這封信很爽:
Dear Dr. Yu：
This message is to confirm that we have received your registration information for the 2007 Far Eastern Meeting of the Econometrics Society.Please check the information appended below for any mistake.
If you have any question regarding the Meeting, please do not hesitate to let us know. We look forward to your participation in the Meeting.
Sincerely yours,2007 FEMES local committee
Sunday, May 27, 2007
Sunday, May 20, 2007
What is new in the world of econ - Part 5: Tax the Tall!
要如何課稅是一個非重要的問題!
課稅本身就是一個違反經濟思考的東西，經濟學講求efficiency，但是課稅很多時後是講求equality。這兩者之間要如何做最適當的trade off是非常重要的課題。有錢人課太多會影響他工作意願，成時報稅，等問題。
這研究從Mirrlees的突破性研究到現在還沒有一個確定的答案。
而最近Mankiw 跟他的學生做了一項非常新鮮的研究:
我非常贊成這個研究的結果!!
課稅本身就是一個違反經濟思考的東西，經濟學講求efficiency，但是課稅很多時後是講求equality。這兩者之間要如何做最適當的trade off是非常重要的課題。有錢人課太多會影響他工作意願，成時報稅，等問題。
這研究從Mirrlees的突破性研究到現在還沒有一個確定的答案。
而最近Mankiw 跟他的學生做了一項非常新鮮的研究:
Mr. Mankiw and Mr. Weinzierl cite their own and others' research that thereis a strong correlation between height and income. By their calculations, a tall person (six feet or higher) earns on average 16 percent more than a short person (five foot nine or less). The authors argue that according totheory, height is a great criterion for income redistribution: tax tallpeople more and give the money to short people.http://www.economics.harvard.edu/faculty/mankiw/papers/Optimal_Taxation.pdf
我非常贊成這個研究的結果!!
Saturday, May 12, 2007
Sunday, May 06, 2007
Econ Concepts: Part 9 - The Predictive Power of NE
NE 是拿來幹麻的? 經濟學家希望透過NE來研究理性的人的策略行為。而NE也的確對經濟學有所貢獻，現代的IO若沒有賽局理論，就少了許多文獻了。
但是顯然的有許多情況下NE會讓我們做出錯誤的判斷。想像下面的對局:
兩個人在第一階段選擇X或Y。若雙方都選擇Y，則遊戲結束，各得一塊錢。若一個選擇Y，另一個選擇X，則遊戲結束，各空手回家。但是若雙方都選擇X，則遊戲到下一個階段，第二個階段，雙方要選擇一個數字，數字較大的人得$250，數字較小的人得100，雙方都一樣的話，各拿100。
Questions:
1. 在這遊戲裡有幾個NE?
2. NE是什麼?
3. 你認為這NE合理嗎? 為什麼?
但是顯然的有許多情況下NE會讓我們做出錯誤的判斷。想像下面的對局:
兩個人在第一階段選擇X或Y。若雙方都選擇Y，則遊戲結束，各得一塊錢。若一個選擇Y，另一個選擇X，則遊戲結束，各空手回家。但是若雙方都選擇X，則遊戲到下一個階段，第二個階段，雙方要選擇一個數字，數字較大的人得$250，數字較小的人得100，雙方都一樣的話，各拿100。
Questions:
1. 在這遊戲裡有幾個NE?
2. NE是什麼?
3. 你認為這NE合理嗎? 為什麼?
Saturday, May 05, 2007
Modern Econmen: Vernon Smith
在網路上找到的影片: 經濟學家還真是diverse
Smith 是實驗經濟學之父:
Smith 是實驗經濟學之父:
"In the Autumn semester, 1955, I taught Principles of Economics, and found it a challenge to convey basic microeconomic theory to students. Why/how could any
market approximate a competitive equilibrium? I resolved that on the first day
of class the following semester, I would try running a market experiment that
would give the students an opportunity to experience an actual market, and me
the opportunity to observe one in which I knew, but they did not know what were
the alleged driving conditions of supply and demand in that market."
Good Books: Game Theory and Economic Modeling
我最近在看一本書: David Kreps 的 Game Theory and Economic Modeling。
這本書探討了Game Theory 對經濟學的貢獻也同時指出他的缺點。主要的重點在於Nash equilibrium的使用是不適合。
我認為所有對經濟學有興趣的人都應該看。沒有學過GT的人也會很喜歡。
這本書探討了Game Theory 對經濟學的貢獻也同時指出他的缺點。主要的重點在於Nash equilibrium的使用是不適合。
我認為所有對經濟學有興趣的人都應該看。沒有學過GT的人也會很喜歡。
Whats in the News: NBA 黑白集
前幾天第五場比賽，小牛隊上勇士，Nowitzki的小牛驚險擊敗勇士隊。勇士的主將Baron Davis因為犯滿畢業，無法幫對有奮戰到最後。
有趣的是，Davis的第六犯是誤判!而吹他犯規的是白人的裁判!我想講什麼呢?NBA的白人裁判歧視黑人!
我不是隨便亂講的，我講的話都是有根據的。最近Wolfers寫了一篇論文，其中就是探討NBA的種族歧視問題。他說:
...during the 13 seasons from 1991 through 2004, white referees called foulsat a greater rate against black players than against white players...found a c\orresponding bias in which black officials called foulsmore frequently against white players, though that tendency was not as strong.
想看全文的人: http://bpp.wharton.upenn.edu/jwolfers/Papers/NBARace.pdf
Levitt說這篇文章找不到任何問題。
最後勇士還是贏了! Yeah! Nowitzki 回家釣魚了!
ps: 若裁判換成我們中國人(黃種人)如何? 可能會更慘，一我的經驗黃種人對黑人的誤解非常深。可能因為接觸不多。可惜! 因為我許多最好的朋友都是黑人。她們人都非常好也很優秀。
有趣的是，Davis的第六犯是誤判!而吹他犯規的是白人的裁判!我想講什麼呢?NBA的白人裁判歧視黑人!
我不是隨便亂講的，我講的話都是有根據的。最近Wolfers寫了一篇論文，其中就是探討NBA的種族歧視問題。他說:
...during the 13 seasons from 1991 through 2004, white referees called foulsat a greater rate against black players than against white players...found a c\orresponding bias in which black officials called foulsmore frequently against white players, though that tendency was not as strong.
想看全文的人: http://bpp.wharton.upenn.edu/jwolfers/Papers/NBARace.pdf
Levitt說這篇文章找不到任何問題。
最後勇士還是贏了! Yeah! Nowitzki 回家釣魚了!
ps: 若裁判換成我們中國人(黃種人)如何? 可能會更慘，一我的經驗黃種人對黑人的誤解非常深。可能因為接觸不多。可惜! 因為我許多最好的朋友都是黑人。她們人都非常好也很優秀。
Tuesday, April 24, 2007
單身漢的天堂! The importance of Common Knowledge
有一個村莊住100對夫妻。村裡的100個男的都會每一星期聚會，若家裡的妻子守貞潔，先生就會誇讚她。若家裡的妻子有外遇，先生就會抱頭痛哭。
但是每一個女的都有外遇，而每一個男的都知道，但是都以為自己的妻子沒有外遇。所以每一星期的聚會都在讚美聲中度過。
直到有一天一位智者路過，在村莊休息。他觀察了一星期後，他決定參加男人每一星期的聚會。在聚會上，他看到所有100個男的讚頌自己的妻子。
他最後就說: "這村裡有一個女的有外遇"。
過了一星期後，所有100個男的繼續讚美自己的妻子。這持續了99星期。但是到了第100個星期，所有的男的都抱頭痛哭。
請問為什麼會這樣?
Saturday, April 21, 2007
Thursday, April 19, 2007
What is New in the World of Econ: Part 4 - 男女大不同
現在的社會強調兩性平等，
但是社會裡的種種現象往往使我們懷疑這機制是否真的平等。
我們是可以懷疑制度或社會機制的問題，
但是若女性跟男性在行為上本來就有很大的差異呢?
這樣一來就很有可能造成看似不"平等"的結果了。
最近有一篇論文做實驗，測試男性與女性在選擇上的差異。
這一篇論文很特別，實驗在經濟學是非常罕見的，但是近幾年來越來越熱門。
他是如何進行實驗呢?:
首先他們將兩個女的跟兩個男的分到一組，
再叫他們做一些簡單的數學運算(國小程度的)當做練習，
練習完後，四個人可以個別單獨選擇玩 "保守遊戲" 或是 "野心遊戲" (詞窮!)
選擇"保守"的人只要答對一題，無論如何都可以拿0.50塊錢。
選擇"野心"的人答對一題拿2塊錢，只有在他是四個人當中答對最多題時才可以拿錢，
若不是選擇野心者就無法拿到錢。
實驗結果發現75%的男性選擇"野心"，而只有百分之25的女性選擇"野心"。
這代表男性高估自己的實力(自以為比全部的人都強)，而女性卻低估。
這結果有許多implication。我想CEO是男性比較多可能跟這有關。
Monday, April 16, 2007
What is New in the World of Econ: Part 3 - Happy Doing Business with You!
曾經想過再哪一個國家經商最便利嗎?World Bank 有做調查，以十個項目當衡量標準:
Starting a Business
Dealing with Licenses
Employing Workers
Registering Property
Getting Credit
Protecting Investors
Paying Taxes
Trading Across Borders
Enforcing Contracts
Closing a Business
以下是經商便利排行的前十名:
Sunday, April 15, 2007
Phenomenons on Planet Econ: Part 3 - Disclosure
這世界很奇妙! 你有可能是一位很勤快的工人，能力像當強，但是除非你真的實際下去工作了，老闆才能知道你的超強能力，所以你有可能面試時就被刷了。這時signaling就派上用場了。
但是若我們今天講的是你去申請保險的事? 要證明你很健康比證明你工作能力強簡單許多，只要拿出體檢結果就行了! 而保險公司要確認也不難。
假設今天政府說個人健康資料是私人資料，因此對方無法迫使你提供你的健康資料。這時就有兩個問題了:
1. 若不再加其他法律規範，這個法條有用嗎?
2. 若要加法條，要加什麼?
1. 這個法條沒有用。因為健康的人還是會提供健康資料，而當你不提供人家就知道你不健康。所以不健康的人還是沒保護到。
2. 若要保護不健康的人，就要全面禁止提供健康資料。
這個例子相當有趣。而他背後的理論就是Grossman and Hart的Full Disclusure Theorem。
但是若我們今天講的是你去申請保險的事? 要證明你很健康比證明你工作能力強簡單許多，只要拿出體檢結果就行了! 而保險公司要確認也不難。
假設今天政府說個人健康資料是私人資料，因此對方無法迫使你提供你的健康資料。這時就有兩個問題了:
1. 若不再加其他法律規範，這個法條有用嗎?
2. 若要加法條，要加什麼?
1. 這個法條沒有用。因為健康的人還是會提供健康資料，而當你不提供人家就知道你不健康。所以不健康的人還是沒保護到。
2. 若要保護不健康的人，就要全面禁止提供健康資料。
這個例子相當有趣。而他背後的理論就是Grossman and Hart的Full Disclusure Theorem。
Tuesday, April 03, 2007
社會學家看經濟學家
經濟學家就是社會學家阿! 是嗎?
沒錯，但是似乎社會學家認為經濟學家跟他們不太像。為何呢? 因為最近有幾位社會學家對經濟學家的生態非常感興趣(若他們認為經濟學家跟他們一樣，就不會研究我們)
其中一位的研究結果顯示:"…most of their knowledge is too abstract to be of much substantive use, andtheir standards of academic rigor may play only a minor role in legitimizingtheir day to day authority."
意思就是經濟學家的研究都太抽象(太數學)，而這麼做只是在建立他們在象牙塔內的勢力一小部分的作為。(我照我自己的意思翻)
另一位柏克萊學者說:"Scientific representations, policy paradigms, and internationallinkages all enter the competitive processes whereby different segmentsand groups in the various national economics professions seek to asserttheir authority on particular jurisdictions (professional, scientific, orpolitical)..."
意思就是"美式"的經濟學已成為影響其他國家的一個重要工具。而經濟學界也利用這一點來擴張自己的勢力。
我對這些東西沒意見，只是覺得很好玩。
沒錯，但是似乎社會學家認為經濟學家跟他們不太像。為何呢? 因為最近有幾位社會學家對經濟學家的生態非常感興趣(若他們認為經濟學家跟他們一樣，就不會研究我們)
其中一位的研究結果顯示:"…most of their knowledge is too abstract to be of much substantive use, andtheir standards of academic rigor may play only a minor role in legitimizingtheir day to day authority."
意思就是經濟學家的研究都太抽象(太數學)，而這麼做只是在建立他們在象牙塔內的勢力一小部分的作為。(我照我自己的意思翻)
另一位柏克萊學者說:"Scientific representations, policy paradigms, and internationallinkages all enter the competitive processes whereby different segmentsand groups in the various national economics professions seek to asserttheir authority on particular jurisdictions (professional, scientific, orpolitical)..."
意思就是"美式"的經濟學已成為影響其他國家的一個重要工具。而經濟學界也利用這一點來擴張自己的勢力。
我對這些東西沒意見，只是覺得很好玩。
Monday, April 02, 2007
有趣的故事
Suppose that every day, ten men go out for beer and the bill for all ten comes to $100.
If they paid their bill the way we pay our taxes, it would go something like this:
The first four men (the poorest) would pay nothing.
The fifth would pay $1.
The sixth would pay $3.
The seventh would pay $7.
The eighth would pay $12.
The ninth would pay $18.
The tenth man (the richest) would pay $59.So, that's what they decided to do.
The ten men drank in the bar every day and seemed quite happy with the arrangement, until one day, the owner threw them a curve. "Since you are all such good customers," he said, "I'm going to reduce the cost of your daily beer by $20." Drinks for the ten now cost just $80.
The group still wanted to pay their bill the way we pay our taxes so the first four men were unaffected. They would still drink for free. But what about the other six men - the paying customers? How could they divide the $20 windfall so that everyone would get his 'fair share?' They realized that $20 divided by six is $3.33. But if they subtracted that from everybody's share, then the fifth man and the sixth man would each end up being paid to drink his beer. So, the bar owner suggested that it would be fair to reduce each man's bill by roughly the same amount, and he proceeded to work out the amounts each should pay. And so:
The fifth man, like the first four, now paid nothing (100% savings).
The sixth now paid $2 instead of $3 (33%savings).
The seventh now pay $5 instead of $7 (28%savings).
The eighth now paid $9 instead of $12 (25% savings).
The ninth now paid $14 instead of $18 (22% savings).
The tenth now paid $49 instead of $59 (16% savings).
Each of the six was better off than before. And the first four continued to drink for free. But once outside the restaurant, the men began to compare their savings.
"I only got a dollar out of the $20," declared the sixth man. He pointed to the tenth man," but he got $10!""Yeah, that's right," exclaimed the fifth man. "I only saved a dollar, too. It's unfair that he got ten times more than I!"
"That's true!" shouted the seventh man. "Why should he get $10 back when I got only two? The wealthy get all the breaks!"
"Wait a minute," yelled the first four men in unison. "We didn't get anything at all. The system exploits the poor!"
The nine men surrounded the tenth and beat him up.
The next night the tenth man didn't show up for drinks, so the nine sat down and had beers without him. But when it came time to pay the bill, they discovered something important. They didn't have enough money between all of them for even half of the bill!
And that, boys and girls, journalists and college professors, is how our tax system works. The people who pay the highest taxes get the most benefit from a tax reduction. Tax them too much, attack them for being wealthy, and they just may not show up anymore. In fact, they might start drinking overseas where the atmosphere is somewhat friendlier.
這故事還蠻有趣的
If they paid their bill the way we pay our taxes, it would go something like this:
The first four men (the poorest) would pay nothing.
The fifth would pay $1.
The sixth would pay $3.
The seventh would pay $7.
The eighth would pay $12.
The ninth would pay $18.
The tenth man (the richest) would pay $59.So, that's what they decided to do.
The ten men drank in the bar every day and seemed quite happy with the arrangement, until one day, the owner threw them a curve. "Since you are all such good customers," he said, "I'm going to reduce the cost of your daily beer by $20." Drinks for the ten now cost just $80.
The group still wanted to pay their bill the way we pay our taxes so the first four men were unaffected. They would still drink for free. But what about the other six men - the paying customers? How could they divide the $20 windfall so that everyone would get his 'fair share?' They realized that $20 divided by six is $3.33. But if they subtracted that from everybody's share, then the fifth man and the sixth man would each end up being paid to drink his beer. So, the bar owner suggested that it would be fair to reduce each man's bill by roughly the same amount, and he proceeded to work out the amounts each should pay. And so:
The fifth man, like the first four, now paid nothing (100% savings).
The sixth now paid $2 instead of $3 (33%savings).
The seventh now pay $5 instead of $7 (28%savings).
The eighth now paid $9 instead of $12 (25% savings).
The ninth now paid $14 instead of $18 (22% savings).
The tenth now paid $49 instead of $59 (16% savings).
Each of the six was better off than before. And the first four continued to drink for free. But once outside the restaurant, the men began to compare their savings.
"I only got a dollar out of the $20," declared the sixth man. He pointed to the tenth man," but he got $10!""Yeah, that's right," exclaimed the fifth man. "I only saved a dollar, too. It's unfair that he got ten times more than I!"
"That's true!" shouted the seventh man. "Why should he get $10 back when I got only two? The wealthy get all the breaks!"
"Wait a minute," yelled the first four men in unison. "We didn't get anything at all. The system exploits the poor!"
The nine men surrounded the tenth and beat him up.
The next night the tenth man didn't show up for drinks, so the nine sat down and had beers without him. But when it came time to pay the bill, they discovered something important. They didn't have enough money between all of them for even half of the bill!
And that, boys and girls, journalists and college professors, is how our tax system works. The people who pay the highest taxes get the most benefit from a tax reduction. Tax them too much, attack them for being wealthy, and they just may not show up anymore. In fact, they might start drinking overseas where the atmosphere is somewhat friendlier.
這故事還蠻有趣的
Saturday, March 17, 2007
Robust Regression
我們新的blogger: Avici 昨天試圖要潑這篇文章但沒成功
The term "robust regression" can be used to mean two slightly different things. The first usage should really be called regression with robust standard errors. In regression with robust standard errors the estimates of the regressioncoefficients are the same as in the standard OLS linear regressionbut the estimates of the standard errors are more robust to failure to meet assumptions concerning normality and homogeneity of variance ofthe residuals. The second way the term robust regression is used involves bothrobust estimation of the regression coefficients and the standard errors. This approach is useful in situations where there are large outliersand observations with large leverage values.
如他所說 這裡的robust事實上有兩種意思:
The first usage should really be called regression with robust standard error
就是你跑完回歸要對估計參數做檢定時 你得知道variance-covariance matrix才行
(底下簡稱VC) 在古典OLS 10條假設下 那很簡單 回歸係數的VC就是s^2*inverse(X'X)
問題是...那10條假設 實際上根本不可能有那麼好的情況XD
如果有變異數不齊一(以後簡稱hetero) 或者自我相關(簡稱auto)
那VC就不是那樣子 可是問題是 沒有人知道真正的hetero 和auto的形式
所以根本沒辦法推導出VC的估計式 此時就只好改用無母數的方法 spectral analysis等
把這些效果考慮進來
第一個做這個問題的就是管老師的老師Halbert White
然後就有所謂VC的估計式robust to conditional hetero
緊接著如果同時有auto加hetero 那又更麻煩了 所以這20年來幾乎所有有名的計量學家都做過這個題目 文獻上稱HAC (hetero-and-auto consistency) VC estimator
所以所謂的robust to hetero-and auto 言下之意就是指
就算有hetero and auto 我的估計依然不受影響
robust一般翻譯做"穩健性"
使用方法很簡單 你一樣是跑個回歸 估計出係數 但是變異數要用HAC就是了
不過這玩意有很多問題低 到現在都還沒完全解決 其實這就是我的碩士論文題目XD
這是個很根本的問題 因為大部分使用的檢定 不是Wald test就是LM test
而這兩者都需要估計VC矩陣 如果連這個都估錯 那所有的推論都是可質疑的
另外robust不是回歸才有 所有統計上的檢定和估計 都有這個問題
就是推導估計檢定的時候 都得作一些假設 有些假設根本和資料特性不合
最經典的莫過於OLS 10條假設...Orz
萬一把這個假設拿掉 我的估計和檢定方式 該怎麼"校正" 才能robust to outlier
這些違反假設條件的資料特性
所以這是很重要的課題 晚近10年 甚至往後的10幾年
把現有的統計通通"robustify"將會是主流研究 也就是把統計推廣到更一般的情況
接著我們來看第二種意義
The second way the term robust regression is used involves both robust estimation of the regression coefficients and the standard errors. This approach is useful in situations where there are large outliers and observations with large leverage values.
一開始研究robust的數學家 就是要看outlier對於回歸係數估計的影響
對於t檢定效率的影響等等
所以一開始robust指的就是 robust to outlier
當然現在robust可以to各式各樣的東西:p
為了搭配你下面一篇 我講一下這幾個的由來 以及為什麼需要這幾個
Median regression,Absolute value regression (L1),
OLS的基本性質 就是minimize sum of square推導出來的
OLS的基本性質 就是minimize sum of square推導出來的
做這件事情 只是一種計算 但如果在隨機的世界 那這個意義就是
你在描繪y的conditional mean, conditional on解釋變數X
所以OLS的基本性質和mean是緊緊相連的
可是mean的基本性質 就是會受極端值影響
解決之道就是 我想找一個也是描述平均性質的統計量 但可不受outlier影響
有哪個統計量具有這種性質 就是median囉
舉例: -1,0,1 -10,0,1 -1000,0,1
這三種不同資料 顯然mean都不一樣 可是median都是一樣
描繪conditional median的估計式就是
min sum(y-a-bx)
所以從這個式子 以及median的特性
才被稱作Median regression,Absolute value regression (L1),
一般其實叫做LAD估計(least absolute deviation)
不過這種估計式的一個特色就是...100% insensitive to outlier
OLS其實是100% sensitive to outlier
所以又有人開發敏感度介於這兩種之間的估計 稱為Huber estimator
R裡面有功能可以求解這種回歸
因為這個不能微分所以頗麻煩 不過可以轉成linear programming的問題來做
當然估計方法換了 VC矩陣也變不一樣了
在這種奇怪的回歸下 VC矩陣到底長什麼樣 該怎麼robustify
都仍在研究當中
The term "robust regression" can be used to mean two slightly different things. The first usage should really be called regression with robust standard errors. In regression with robust standard errors the estimates of the regressioncoefficients are the same as in the standard OLS linear regressionbut the estimates of the standard errors are more robust to failure to meet assumptions concerning normality and homogeneity of variance ofthe residuals. The second way the term robust regression is used involves bothrobust estimation of the regression coefficients and the standard errors. This approach is useful in situations where there are large outliersand observations with large leverage values.
如他所說 這裡的robust事實上有兩種意思:
The first usage should really be called regression with robust standard error
就是你跑完回歸要對估計參數做檢定時 你得知道variance-covariance matrix才行
(底下簡稱VC) 在古典OLS 10條假設下 那很簡單 回歸係數的VC就是s^2*inverse(X'X)
問題是...那10條假設 實際上根本不可能有那麼好的情況XD
如果有變異數不齊一(以後簡稱hetero) 或者自我相關(簡稱auto)
那VC就不是那樣子 可是問題是 沒有人知道真正的hetero 和auto的形式
所以根本沒辦法推導出VC的估計式 此時就只好改用無母數的方法 spectral analysis等
把這些效果考慮進來
第一個做這個問題的就是管老師的老師Halbert White
然後就有所謂VC的估計式robust to conditional hetero
緊接著如果同時有auto加hetero 那又更麻煩了 所以這20年來幾乎所有有名的計量學家都做過這個題目 文獻上稱HAC (hetero-and-auto consistency) VC estimator
所以所謂的robust to hetero-and auto 言下之意就是指
就算有hetero and auto 我的估計依然不受影響
robust一般翻譯做"穩健性"
使用方法很簡單 你一樣是跑個回歸 估計出係數 但是變異數要用HAC就是了
不過這玩意有很多問題低 到現在都還沒完全解決 其實這就是我的碩士論文題目XD
這是個很根本的問題 因為大部分使用的檢定 不是Wald test就是LM test
而這兩者都需要估計VC矩陣 如果連這個都估錯 那所有的推論都是可質疑的
另外robust不是回歸才有 所有統計上的檢定和估計 都有這個問題
就是推導估計檢定的時候 都得作一些假設 有些假設根本和資料特性不合
最經典的莫過於OLS 10條假設...Orz
萬一把這個假設拿掉 我的估計和檢定方式 該怎麼"校正" 才能robust to outlier
這些違反假設條件的資料特性
所以這是很重要的課題 晚近10年 甚至往後的10幾年
把現有的統計通通"robustify"將會是主流研究 也就是把統計推廣到更一般的情況
接著我們來看第二種意義
The second way the term robust regression is used involves both robust estimation of the regression coefficients and the standard errors. This approach is useful in situations where there are large outliers and observations with large leverage values.
一開始研究robust的數學家 就是要看outlier對於回歸係數估計的影響
對於t檢定效率的影響等等
所以一開始robust指的就是 robust to outlier
當然現在robust可以to各式各樣的東西:p
為了搭配你下面一篇 我講一下這幾個的由來 以及為什麼需要這幾個
Median regression,Absolute value regression (L1),
OLS的基本性質 就是minimize sum of square推導出來的
OLS的基本性質 就是minimize sum of square推導出來的
做這件事情 只是一種計算 但如果在隨機的世界 那這個意義就是
你在描繪y的conditional mean, conditional on解釋變數X
所以OLS的基本性質和mean是緊緊相連的
可是mean的基本性質 就是會受極端值影響
解決之道就是 我想找一個也是描述平均性質的統計量 但可不受outlier影響
有哪個統計量具有這種性質 就是median囉
舉例: -1,0,1 -10,0,1 -1000,0,1
這三種不同資料 顯然mean都不一樣 可是median都是一樣
描繪conditional median的估計式就是
min sum(y-a-bx)
所以從這個式子 以及median的特性
才被稱作Median regression,Absolute value regression (L1),
一般其實叫做LAD估計(least absolute deviation)
不過這種估計式的一個特色就是...100% insensitive to outlier
OLS其實是100% sensitive to outlier
所以又有人開發敏感度介於這兩種之間的估計 稱為Huber estimator
R裡面有功能可以求解這種回歸
因為這個不能微分所以頗麻煩 不過可以轉成linear programming的問題來做
當然估計方法換了 VC矩陣也變不一樣了
在這種奇怪的回歸下 VC矩陣到底長什麼樣 該怎麼robustify
都仍在研究當中
Subscribe to:
Comments (Atom)
