SlideShare ist ein Scribd-Unternehmen logo
1 von 31
Catch Me If You Can
Customer Fund Bug Analysis
Liang Gao
Analysis Customer Found Bug is Good
• Why we didn’t find it through our internal
testing
• What test case can be designed to catch that
• What kind of test strategy can cover that
• How can we make sure we can catch this kind
if bug from now on
Bug # 1, WebEx Bug:
• In Windows, if you share Adobe Acrobat (PDF)
files in landscape mode, they may display in
portrait mode
• In Mac, You can only connect to WebEx sessions
from behind a Microsoft ISA proxy server, in basic
mode, that has user authentication enabled.
• In Linux: you cannot clear just your own
annotations. When you clear annotations, all
annotations are removed.
Bug # 2 WebEx Bug:
• If Active X is disabled in Internet Explorer,
contacts cannot be imported from Microsoft
Outlook.
• In Mac, You can only connect to WebEx sessions
from behind a Microsoft ISA proxy server, in basic
mode, that has user authentication enabled.
• In Linux: you cannot clear just your own
annotations. When you clear annotations, all
annotations are removed.
Bug # 3 WebEx Bug:
• If a single occurrence of a recurring WebEx meeting is
either deleted or rescheduled, the meeting
information is not updated on the WebEx service site.
In the host and attendee's Outlook calendars,
however, the deleted or rescheduled meeting still
appears correctly.
• If a template used during Outlook integration has
"Mute on Entry" option enabled, you will still hear a
sound as attendees join the session.
• Attendee registration can not be enabled for recurring
WebEx meetings scheduled using Lotus Notes
Integration.
Bug # 4 Taobao Bug:
Bug # 5 Taobao Bug:
http://wuliu.taobao.com/user/order_list_new.htm?order_status_show=monyer”>
<img%20src=http://www.baidu.com/img/baidu_logo.gif%20onload=
Bug # 6 Alisoft Bug:
http://wuliu.taobao.com/user/order_list_new.htm?order_status_show=monyer”>
<img%20src=http://www.baidu.com/img/baidu_logo.gif%20onload=
http://webwwtb2.im.alisoft.com/wangwang/ww1.htm?t=1222861728484&uid=monye
m%3Cimg%20src=javascript:alert("monyertest"+document.cookie)%3E
Bug # Bug 7 Taobao Bug:
http://wuliu.taobao.com/user/order_list_new.htm?order_status_show=monyer”>
<img%20src=http://www.baidu.com/img/baidu_logo.gif%20onload=
http://webwwtb2.im.alisoft.com/wangwang/ww1.htm?t=1222861728484&uid=monye
m%3Cimg%20src=javascript:alert("monyertest"+document.cookie)%3E
http://upload.taobao.com/auction/publish/publish.htm?auction_type=monyer"%3E%
Cimg%20src=http://www.baidu.com/img/baidu_logo.gif%3E
Bug 9: Google Doc Sharing Bug
We have two documents with one owner and two contributors each:
Document 1, contributors: A, B
Document 2, contributors: C, D
If I were to select both documents and make E a contributor,
this is what I would expect to happen:
Document 1, contributors: A, B, E
Document 2, contributors: C, D, E
This is what actually happened:
Document 1, contributors: A, B, C, D, E
Document 2, contributors: C, D, E
Bug 10: Google Doc Authentication
Bug
For Google Doc,
an image embedded into
a protected document is given a URL which is not protected
Bug 11: Office Online Bug
Office Online
Bug
Bug 12: Boundary Testing Bugs
14
 214-748-3647
Most popular
phone number
in US
 Largest 32 bit
signed number
 Store phone
number in a
signed 32 bits
and didn’t check
buffer overflow
Bug 13: Visa Credit Card Bug
Recently several Visa card holders were overcharged for certain purchases,
to the tune of $23,148,855,308,184,500.00 on a single charge.
The company says it was due to a programming error, and that the problem
has been corrected.
What is interesting is that the amount charged actually reveals the type of
programming error that caused the problem. 23,148,855,308,184,500.00 *
100 (I'm guessing this is how the number is actually stored) is
2314885530818450000. Convert 2314885530818450000 to hexadecimal,
and you end up with 20 20 20 20 20 20 12 50. Most C/C++ programmers see
the error now ... hex 20 is a space. So spaces were stuffed into a field where
binary zero should have been."
Bug 16: Cisco Bug
• Title: 在向某防火墙发送 version 字段为 0 的
IPv6 报文时,打开防火墙的 snoop ,会造
成防火墙重启 .
• How would you design test case?
• Why it was not caught internally
• What kind of test strategy can cover this?
Bug 17: Cisco Bug
• 处理 IPv6 分片 ICMP 大包 . 防火墙上结果是
未通过
• How would you design test case?
• Why it was not caught internally
• What kind of test strategy can cover this?
Bug 18: Cisco Bug
• 某网络安全代理产品:当访问已有代理的
Web 服务器时候访问不了
• How would you design test case?
• Why it was not caught internally
• What kind of test strategy can cover this?Content secure gateway
Proxy Web Server
Bug 19: Cisco Bug
• 配置了 65535 个 RP 和 1785 个 vlan 的 IP
地址后, wr ,死机,重新断电启动,
等待 10 分钟后仍然无法启动
• How would you design test case?
• Why it was not caught internally
• What kind of test strategy can cover this?
Bug 20: Cisco Bug
• 当使用 BGP PEER GROUP 时,当邻居
实际 AS 与配置的 AS 不同时,仍能建
立连接
Bug 21: 银行
• 网上银行使用银联来做认证
• 银联升级, 凌晨
• 15 分钟之内所有银联的认证全部默认
通过
• 所有网上银行交易(网购等) 15 分钟
内无需密码(任意密码)就可以成功
Bug 22: 银行
• 外汇交易,汇率信息来自路透社
• 路透社和北京时间有时差,有一段时
间不会有信息更新
• 系统实现时,如果没有信息更新,使
用缺省的汇率
• 被客户发现并利用,损失了上百万
Bug 24: 网络• 瑞典因例行维护时造成 DNS 不能识别域名中的“ .se” 而使全瑞典互联网断网
近一个小时。
•
•       瑞典当地时间 2009 年 10 月 12 日晚上 9 时 45 分,全瑞典所有网站无法连接,
所有带瑞典域的电子邮件都无法正常接收和发送,有大约 90 万域名受到影响。
•
•       瑞典网络监控公司 Pingdom 指出 , 在对“ .se” 域升级时的“脚本配置错误”是引起
这次网络故障的原因。
•
•       很显然,程序末尾仅少了个句号使得域名系统( DNS )无法识
别“ .se ” 了,“ .se” 是瑞典的“顶级”(国家)域。(译注:“ se” 取自  Sweden ,就
像“ cn” 取自中国 china 一样)
•       在对脚本测试期间,这个遗漏的句号没有被发现。而该软件一旦投入运行,
监视系统便发现该遗漏的句号,并生成一个新文件。
•
•       然而,由于旧脚本信息缓存在各互联网服务提供商( ISP )中,要等到各 ISP
重新还原系统,由新脚本引起的中断才告结束。修正后的新脚本在当地时间晚上
10 时 43 分投入运行。
Bug 25: 携程
• 点数换机票,需要上网认证,并通过
手机认证,客户收到认证码后,再上
网确认。
• 里程部看到的是里程数已经可以用了
• 而客服部看到的是还不能够购买
• 客户在这两个部门间被踢来踢去,一
个很好的客户满意计划变成了客户抱
怨计划。
Bug 26: 意大利邮电局系统更新
Bug• 2009 年 11 月 25 日邮局系统更新,包
括所有的 ATM
• 整数后面的小数点被去掉,取 115.00
欧元被认为是取了 11500 欧元
• 上万人的账户显示透支,不能再使用
。
• 客服电话被打爆
Bug 27: 微软 Office 2003 权限
Bug• Cannot Open Office 2003 Documents
Protected with RMS
• Starting on December 11, 2009, customers
using Office 2003 will not be able to open
Office 2003 documents protected with the
Rights Management Service (RMS) or save
Office 2003 documents protected with
RMS. The following error message may be
displayed when attempting to Open RMS
Documents using Office 2003:
Bug 28: 微软手机 Bug
• Messages received after 1/1/2010 may be
dated as 2016
• Today's date 010110
• BCD 10 is 0001 0000 in binary, which is 16
in decimal.
• Bank of Queensland’s Eftpos terminals.
OQ’s Eftpos machines skipped ahead six
years when the clock ticked over to January
1 and started date stamping January 2016.
Bug 29: SpamAssassin Bug
• Messages received after 1/1/2010 are all
treated as Spam
• Promptly at the start of the new year, all
mails started getting an extra 3.4 points
based on FH_DATE_PAST_20XX:
header FH_DATE_PAST_20XX Date
=~ /20[1-9][0-9]/ [if-unset:
2006]
describe FH_DATE_PAST_20XX
The date is grossly in the
future.
Bug 30: Mars Pathfinder Bug
• 2+2 = 5 check
• 一个产生偶数的算法
• 实验室中只发生了一次,无法重现
• 中断发生了一次,在执行算法之前(百
万分之一秒)
Bug 的修复费用从顶层到底层逐
渐增多
Why we didn't catch that   application bugs

Weitere ähnliche Inhalte

Was ist angesagt?

VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC
VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC
VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC VMworld
 
Oracle forms and reports 11g installation on linux
Oracle forms and reports 11g installation on linuxOracle forms and reports 11g installation on linux
Oracle forms and reports 11g installation on linuxVenu Palakolanu
 
VMworld 2015: Managing vSphere 6 Deployments and Upgrades
VMworld 2015: Managing vSphere 6 Deployments and Upgrades VMworld 2015: Managing vSphere 6 Deployments and Upgrades
VMworld 2015: Managing vSphere 6 Deployments and Upgrades VMworld
 
Planning & Completing An IBM Connections Upgrade
Planning & Completing An IBM Connections UpgradePlanning & Completing An IBM Connections Upgrade
Planning & Completing An IBM Connections UpgradeGabriella Davis
 
156 816
156 816156 816
156 816edfina
 
Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...
Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...
Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...Atlassian
 
086 Microsoft Application Platform 2009 2010
086 Microsoft Application Platform 2009 2010086 Microsoft Application Platform 2009 2010
086 Microsoft Application Platform 2009 2010GeneXus
 
Debugging with Fiddler
Debugging with FiddlerDebugging with Fiddler
Debugging with FiddlerIdo Flatow
 
Sneak Peek into the New ChangeMan ZMF Release
Sneak Peek into the New ChangeMan ZMF ReleaseSneak Peek into the New ChangeMan ZMF Release
Sneak Peek into the New ChangeMan ZMF ReleaseNavita Sood
 
CampusSDN2017 - Jawdat: SDN Technology Evolvement
CampusSDN2017 - Jawdat: SDN Technology EvolvementCampusSDN2017 - Jawdat: SDN Technology Evolvement
CampusSDN2017 - Jawdat: SDN Technology EvolvementJawdatTI
 
Workbook dtp added ccnp enterprise workbook v1.0
Workbook dtp added   ccnp enterprise workbook v1.0Workbook dtp added   ccnp enterprise workbook v1.0
Workbook dtp added ccnp enterprise workbook v1.0SagarR24
 
CCNA Security - Chapter 4
CCNA Security - Chapter 4CCNA Security - Chapter 4
CCNA Security - Chapter 4Irsandi Hasan
 
The Tantric Team: Getting Your Automated Build Groove On
The Tantric Team: Getting Your Automated Build Groove OnThe Tantric Team: Getting Your Automated Build Groove On
The Tantric Team: Getting Your Automated Build Groove OnAtlassian
 
Networking Layer Basics - ChinaNetCloud Training
Networking Layer Basics - ChinaNetCloud TrainingNetworking Layer Basics - ChinaNetCloud Training
Networking Layer Basics - ChinaNetCloud TrainingChinaNetCloud
 

Was ist angesagt? (20)

VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC
VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC
VMworld 2013: How to Exchange Status Message Between Guest and Host Using RPC
 
Oracle forms and reports 11g installation on linux
Oracle forms and reports 11g installation on linuxOracle forms and reports 11g installation on linux
Oracle forms and reports 11g installation on linux
 
VMworld 2015: Managing vSphere 6 Deployments and Upgrades
VMworld 2015: Managing vSphere 6 Deployments and Upgrades VMworld 2015: Managing vSphere 6 Deployments and Upgrades
VMworld 2015: Managing vSphere 6 Deployments and Upgrades
 
Planning & Completing An IBM Connections Upgrade
Planning & Completing An IBM Connections UpgradePlanning & Completing An IBM Connections Upgrade
Planning & Completing An IBM Connections Upgrade
 
156 816
156 816156 816
156 816
 
cudbardbell-freetheradius
cudbardbell-freetheradiuscudbardbell-freetheradius
cudbardbell-freetheradius
 
Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...
Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...
Peer Code Review: In a Nutshell and The Tantric Team: Getting Your Automated ...
 
Iuwne10 S06 L03
Iuwne10 S06 L03Iuwne10 S06 L03
Iuwne10 S06 L03
 
Read me
Read meRead me
Read me
 
086 Microsoft Application Platform 2009 2010
086 Microsoft Application Platform 2009 2010086 Microsoft Application Platform 2009 2010
086 Microsoft Application Platform 2009 2010
 
ClearPass 6.3.6 Release Notes
ClearPass 6.3.6 Release NotesClearPass 6.3.6 Release Notes
ClearPass 6.3.6 Release Notes
 
Debugging with Fiddler
Debugging with FiddlerDebugging with Fiddler
Debugging with Fiddler
 
Iuwne10 S06 L01
Iuwne10 S06 L01Iuwne10 S06 L01
Iuwne10 S06 L01
 
Sa mog
Sa mogSa mog
Sa mog
 
Sneak Peek into the New ChangeMan ZMF Release
Sneak Peek into the New ChangeMan ZMF ReleaseSneak Peek into the New ChangeMan ZMF Release
Sneak Peek into the New ChangeMan ZMF Release
 
CampusSDN2017 - Jawdat: SDN Technology Evolvement
CampusSDN2017 - Jawdat: SDN Technology EvolvementCampusSDN2017 - Jawdat: SDN Technology Evolvement
CampusSDN2017 - Jawdat: SDN Technology Evolvement
 
Workbook dtp added ccnp enterprise workbook v1.0
Workbook dtp added   ccnp enterprise workbook v1.0Workbook dtp added   ccnp enterprise workbook v1.0
Workbook dtp added ccnp enterprise workbook v1.0
 
CCNA Security - Chapter 4
CCNA Security - Chapter 4CCNA Security - Chapter 4
CCNA Security - Chapter 4
 
The Tantric Team: Getting Your Automated Build Groove On
The Tantric Team: Getting Your Automated Build Groove OnThe Tantric Team: Getting Your Automated Build Groove On
The Tantric Team: Getting Your Automated Build Groove On
 
Networking Layer Basics - ChinaNetCloud Training
Networking Layer Basics - ChinaNetCloud TrainingNetworking Layer Basics - ChinaNetCloud Training
Networking Layer Basics - ChinaNetCloud Training
 

Ähnlich wie Why we didn't catch that application bugs

Malware Analysis For The Enterprise
Malware Analysis For The EnterpriseMalware Analysis For The Enterprise
Malware Analysis For The EnterpriseJason Ross
 
Usersnap and the javascript magic behind the scenes - ViennaJS
Usersnap and the javascript magic behind the scenes - ViennaJSUsersnap and the javascript magic behind the scenes - ViennaJS
Usersnap and the javascript magic behind the scenes - ViennaJSUsersnap
 
Modern UI Development With Node.js
Modern UI Development With Node.jsModern UI Development With Node.js
Modern UI Development With Node.jsRyan Anklam
 
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsYevgeniy Brikman
 
The Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptx
The Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptxThe Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptx
The Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptxlior mazor
 
DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...
DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...
DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...Felipe Prado
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryPriyanka Aash
 
Oscon 2011-mueller-weinre
Oscon 2011-mueller-weinreOscon 2011-mueller-weinre
Oscon 2011-mueller-weinrepmuellr
 
Here Be Dragons – Advanced JavaScript Debugging
Here Be Dragons – Advanced JavaScript DebuggingHere Be Dragons – Advanced JavaScript Debugging
Here Be Dragons – Advanced JavaScript DebuggingFITC
 
FITC - Here Be Dragons: Advanced JavaScript Debugging
FITC - Here Be Dragons: Advanced JavaScript DebuggingFITC - Here Be Dragons: Advanced JavaScript Debugging
FITC - Here Be Dragons: Advanced JavaScript DebuggingRami Sayar
 
StHack 2014 - Jerome "@funoverip" Nokin Turning your managed av into my botnet
StHack 2014 - Jerome "@funoverip" Nokin  Turning your managed av into my botnetStHack 2014 - Jerome "@funoverip" Nokin  Turning your managed av into my botnet
StHack 2014 - Jerome "@funoverip" Nokin Turning your managed av into my botnetStHack
 
Increasing velocity via serless semantics
Increasing velocity via serless semanticsIncreasing velocity via serless semantics
Increasing velocity via serless semanticsKfir Bloch
 
How to secure your web applications with NGINX
How to secure your web applications with NGINXHow to secure your web applications with NGINX
How to secure your web applications with NGINXWallarm
 
VMworld 2013: VMware Horizon View Troubleshooting: Looking under the Hood
VMworld 2013: VMware Horizon View Troubleshooting: Looking under the HoodVMworld 2013: VMware Horizon View Troubleshooting: Looking under the Hood
VMworld 2013: VMware Horizon View Troubleshooting: Looking under the HoodVMworld
 
Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016
Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016
Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016Gunawan T Wicaksono
 
Node Js Websocket Js Meetup Slides
Node Js Websocket Js Meetup SlidesNode Js Websocket Js Meetup Slides
Node Js Websocket Js Meetup SlidesMakoto Inoue
 
Fundamentals of network hacking
Fundamentals of network hackingFundamentals of network hacking
Fundamentals of network hackingPranshu Pareek
 

Ähnlich wie Why we didn't catch that application bugs (20)

Malware Analysis For The Enterprise
Malware Analysis For The EnterpriseMalware Analysis For The Enterprise
Malware Analysis For The Enterprise
 
Usersnap and the javascript magic behind the scenes - ViennaJS
Usersnap and the javascript magic behind the scenes - ViennaJSUsersnap and the javascript magic behind the scenes - ViennaJS
Usersnap and the javascript magic behind the scenes - ViennaJS
 
Modern UI Development With Node.js
Modern UI Development With Node.jsModern UI Development With Node.js
Modern UI Development With Node.js
 
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
 
Surge2012
Surge2012Surge2012
Surge2012
 
The Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptx
The Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptxThe Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptx
The Hacking Games - A Road to Post Exploitation Meetup - 20240222.pptx
 
Mini-Training Owin Katana
Mini-Training Owin KatanaMini-Training Owin Katana
Mini-Training Owin Katana
 
DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...
DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...
DEF CON 27 - ORANGE TSAI and MEH CHANG - infiltrating corporate intranet like...
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec glory
 
Oscon 2011-mueller-weinre
Oscon 2011-mueller-weinreOscon 2011-mueller-weinre
Oscon 2011-mueller-weinre
 
Here Be Dragons – Advanced JavaScript Debugging
Here Be Dragons – Advanced JavaScript DebuggingHere Be Dragons – Advanced JavaScript Debugging
Here Be Dragons – Advanced JavaScript Debugging
 
FITC - Here Be Dragons: Advanced JavaScript Debugging
FITC - Here Be Dragons: Advanced JavaScript DebuggingFITC - Here Be Dragons: Advanced JavaScript Debugging
FITC - Here Be Dragons: Advanced JavaScript Debugging
 
StHack 2014 - Jerome "@funoverip" Nokin Turning your managed av into my botnet
StHack 2014 - Jerome "@funoverip" Nokin  Turning your managed av into my botnetStHack 2014 - Jerome "@funoverip" Nokin  Turning your managed av into my botnet
StHack 2014 - Jerome "@funoverip" Nokin Turning your managed av into my botnet
 
ServerSentEventsV2.pdf
ServerSentEventsV2.pdfServerSentEventsV2.pdf
ServerSentEventsV2.pdf
 
Increasing velocity via serless semantics
Increasing velocity via serless semanticsIncreasing velocity via serless semantics
Increasing velocity via serless semantics
 
How to secure your web applications with NGINX
How to secure your web applications with NGINXHow to secure your web applications with NGINX
How to secure your web applications with NGINX
 
VMworld 2013: VMware Horizon View Troubleshooting: Looking under the Hood
VMworld 2013: VMware Horizon View Troubleshooting: Looking under the HoodVMworld 2013: VMware Horizon View Troubleshooting: Looking under the Hood
VMworld 2013: VMware Horizon View Troubleshooting: Looking under the Hood
 
Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016
Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016
Open Mic Webcast: IBM Sametime Audio Video Troubleshooting - 04 May 2016
 
Node Js Websocket Js Meetup Slides
Node Js Websocket Js Meetup SlidesNode Js Websocket Js Meetup Slides
Node Js Websocket Js Meetup Slides
 
Fundamentals of network hacking
Fundamentals of network hackingFundamentals of network hacking
Fundamentals of network hacking
 

Mehr von gaoliang641

Understand release engineering
Understand release engineeringUnderstand release engineering
Understand release engineeringgaoliang641
 
Understand regression testing
Understand regression testingUnderstand regression testing
Understand regression testinggaoliang641
 
The art of system and solution testing
The art of system and solution testingThe art of system and solution testing
The art of system and solution testinggaoliang641
 
Tester performance evaluation
Tester performance evaluationTester performance evaluation
Tester performance evaluationgaoliang641
 
Tester developer interaction
Tester developer interactionTester developer interaction
Tester developer interactiongaoliang641
 
Tester career path
Tester career pathTester career path
Tester career pathgaoliang641
 
Agile testing for large projects
Agile testing for large projectsAgile testing for large projects
Agile testing for large projectsgaoliang641
 
Project management for qa manager
Project management for qa managerProject management for qa manager
Project management for qa managergaoliang641
 
Make good use of explortary testing
Make good use of explortary testingMake good use of explortary testing
Make good use of explortary testinggaoliang641
 
Bug best practice
Bug best practiceBug best practice
Bug best practicegaoliang641
 
Lessons learned on localization testing
Lessons learned on localization testingLessons learned on localization testing
Lessons learned on localization testinggaoliang641
 
Lessons learned on software testing automation
Lessons learned on software testing automationLessons learned on software testing automation
Lessons learned on software testing automationgaoliang641
 
How to become a testing expert
How to become a testing expertHow to become a testing expert
How to become a testing expertgaoliang641
 
Functionality testing techniqu
Functionality testing techniquFunctionality testing techniqu
Functionality testing techniqugaoliang641
 
Protocol Security Testing best practice
Protocol Security Testing best practiceProtocol Security Testing best practice
Protocol Security Testing best practicegaoliang641
 
Backward thinking design qa system for quality goals
Backward thinking   design qa system for quality goalsBackward thinking   design qa system for quality goals
Backward thinking design qa system for quality goalsgaoliang641
 
Automation framework design and implementation
Automation framework design and implementationAutomation framework design and implementation
Automation framework design and implementationgaoliang641
 
Automation from start to finish
Automation   from start to finishAutomation   from start to finish
Automation from start to finishgaoliang641
 
Agile testing for large projects
Agile testing for large projectsAgile testing for large projects
Agile testing for large projectsgaoliang641
 

Mehr von gaoliang641 (19)

Understand release engineering
Understand release engineeringUnderstand release engineering
Understand release engineering
 
Understand regression testing
Understand regression testingUnderstand regression testing
Understand regression testing
 
The art of system and solution testing
The art of system and solution testingThe art of system and solution testing
The art of system and solution testing
 
Tester performance evaluation
Tester performance evaluationTester performance evaluation
Tester performance evaluation
 
Tester developer interaction
Tester developer interactionTester developer interaction
Tester developer interaction
 
Tester career path
Tester career pathTester career path
Tester career path
 
Agile testing for large projects
Agile testing for large projectsAgile testing for large projects
Agile testing for large projects
 
Project management for qa manager
Project management for qa managerProject management for qa manager
Project management for qa manager
 
Make good use of explortary testing
Make good use of explortary testingMake good use of explortary testing
Make good use of explortary testing
 
Bug best practice
Bug best practiceBug best practice
Bug best practice
 
Lessons learned on localization testing
Lessons learned on localization testingLessons learned on localization testing
Lessons learned on localization testing
 
Lessons learned on software testing automation
Lessons learned on software testing automationLessons learned on software testing automation
Lessons learned on software testing automation
 
How to become a testing expert
How to become a testing expertHow to become a testing expert
How to become a testing expert
 
Functionality testing techniqu
Functionality testing techniquFunctionality testing techniqu
Functionality testing techniqu
 
Protocol Security Testing best practice
Protocol Security Testing best practiceProtocol Security Testing best practice
Protocol Security Testing best practice
 
Backward thinking design qa system for quality goals
Backward thinking   design qa system for quality goalsBackward thinking   design qa system for quality goals
Backward thinking design qa system for quality goals
 
Automation framework design and implementation
Automation framework design and implementationAutomation framework design and implementation
Automation framework design and implementation
 
Automation from start to finish
Automation   from start to finishAutomation   from start to finish
Automation from start to finish
 
Agile testing for large projects
Agile testing for large projectsAgile testing for large projects
Agile testing for large projects
 

Kürzlich hochgeladen

Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 

Kürzlich hochgeladen (20)

Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 

Why we didn't catch that application bugs

  • 1. Catch Me If You Can Customer Fund Bug Analysis Liang Gao
  • 2.
  • 3. Analysis Customer Found Bug is Good • Why we didn’t find it through our internal testing • What test case can be designed to catch that • What kind of test strategy can cover that • How can we make sure we can catch this kind if bug from now on
  • 4. Bug # 1, WebEx Bug: • In Windows, if you share Adobe Acrobat (PDF) files in landscape mode, they may display in portrait mode • In Mac, You can only connect to WebEx sessions from behind a Microsoft ISA proxy server, in basic mode, that has user authentication enabled. • In Linux: you cannot clear just your own annotations. When you clear annotations, all annotations are removed.
  • 5. Bug # 2 WebEx Bug: • If Active X is disabled in Internet Explorer, contacts cannot be imported from Microsoft Outlook. • In Mac, You can only connect to WebEx sessions from behind a Microsoft ISA proxy server, in basic mode, that has user authentication enabled. • In Linux: you cannot clear just your own annotations. When you clear annotations, all annotations are removed.
  • 6. Bug # 3 WebEx Bug: • If a single occurrence of a recurring WebEx meeting is either deleted or rescheduled, the meeting information is not updated on the WebEx service site. In the host and attendee's Outlook calendars, however, the deleted or rescheduled meeting still appears correctly. • If a template used during Outlook integration has "Mute on Entry" option enabled, you will still hear a sound as attendees join the session. • Attendee registration can not be enabled for recurring WebEx meetings scheduled using Lotus Notes Integration.
  • 7. Bug # 4 Taobao Bug:
  • 8. Bug # 5 Taobao Bug: http://wuliu.taobao.com/user/order_list_new.htm?order_status_show=monyer”> <img%20src=http://www.baidu.com/img/baidu_logo.gif%20onload=
  • 9. Bug # 6 Alisoft Bug: http://wuliu.taobao.com/user/order_list_new.htm?order_status_show=monyer”> <img%20src=http://www.baidu.com/img/baidu_logo.gif%20onload= http://webwwtb2.im.alisoft.com/wangwang/ww1.htm?t=1222861728484&uid=monye m%3Cimg%20src=javascript:alert("monyertest"+document.cookie)%3E
  • 10. Bug # Bug 7 Taobao Bug: http://wuliu.taobao.com/user/order_list_new.htm?order_status_show=monyer”> <img%20src=http://www.baidu.com/img/baidu_logo.gif%20onload= http://webwwtb2.im.alisoft.com/wangwang/ww1.htm?t=1222861728484&uid=monye m%3Cimg%20src=javascript:alert("monyertest"+document.cookie)%3E http://upload.taobao.com/auction/publish/publish.htm?auction_type=monyer"%3E% Cimg%20src=http://www.baidu.com/img/baidu_logo.gif%3E
  • 11. Bug 9: Google Doc Sharing Bug We have two documents with one owner and two contributors each: Document 1, contributors: A, B Document 2, contributors: C, D If I were to select both documents and make E a contributor, this is what I would expect to happen: Document 1, contributors: A, B, E Document 2, contributors: C, D, E This is what actually happened: Document 1, contributors: A, B, C, D, E Document 2, contributors: C, D, E
  • 12. Bug 10: Google Doc Authentication Bug For Google Doc, an image embedded into a protected document is given a URL which is not protected
  • 13. Bug 11: Office Online Bug Office Online Bug
  • 14. Bug 12: Boundary Testing Bugs 14  214-748-3647 Most popular phone number in US  Largest 32 bit signed number  Store phone number in a signed 32 bits and didn’t check buffer overflow
  • 15. Bug 13: Visa Credit Card Bug Recently several Visa card holders were overcharged for certain purchases, to the tune of $23,148,855,308,184,500.00 on a single charge. The company says it was due to a programming error, and that the problem has been corrected. What is interesting is that the amount charged actually reveals the type of programming error that caused the problem. 23,148,855,308,184,500.00 * 100 (I'm guessing this is how the number is actually stored) is 2314885530818450000. Convert 2314885530818450000 to hexadecimal, and you end up with 20 20 20 20 20 20 12 50. Most C/C++ programmers see the error now ... hex 20 is a space. So spaces were stuffed into a field where binary zero should have been."
  • 16. Bug 16: Cisco Bug • Title: 在向某防火墙发送 version 字段为 0 的 IPv6 报文时,打开防火墙的 snoop ,会造 成防火墙重启 . • How would you design test case? • Why it was not caught internally • What kind of test strategy can cover this?
  • 17. Bug 17: Cisco Bug • 处理 IPv6 分片 ICMP 大包 . 防火墙上结果是 未通过 • How would you design test case? • Why it was not caught internally • What kind of test strategy can cover this?
  • 18. Bug 18: Cisco Bug • 某网络安全代理产品:当访问已有代理的 Web 服务器时候访问不了 • How would you design test case? • Why it was not caught internally • What kind of test strategy can cover this?Content secure gateway Proxy Web Server
  • 19. Bug 19: Cisco Bug • 配置了 65535 个 RP 和 1785 个 vlan 的 IP 地址后, wr ,死机,重新断电启动, 等待 10 分钟后仍然无法启动 • How would you design test case? • Why it was not caught internally • What kind of test strategy can cover this?
  • 20. Bug 20: Cisco Bug • 当使用 BGP PEER GROUP 时,当邻居 实际 AS 与配置的 AS 不同时,仍能建 立连接
  • 21. Bug 21: 银行 • 网上银行使用银联来做认证 • 银联升级, 凌晨 • 15 分钟之内所有银联的认证全部默认 通过 • 所有网上银行交易(网购等) 15 分钟 内无需密码(任意密码)就可以成功
  • 22. Bug 22: 银行 • 外汇交易,汇率信息来自路透社 • 路透社和北京时间有时差,有一段时 间不会有信息更新 • 系统实现时,如果没有信息更新,使 用缺省的汇率 • 被客户发现并利用,损失了上百万
  • 23. Bug 24: 网络• 瑞典因例行维护时造成 DNS 不能识别域名中的“ .se” 而使全瑞典互联网断网 近一个小时。 • •       瑞典当地时间 2009 年 10 月 12 日晚上 9 时 45 分,全瑞典所有网站无法连接, 所有带瑞典域的电子邮件都无法正常接收和发送,有大约 90 万域名受到影响。 • •       瑞典网络监控公司 Pingdom 指出 , 在对“ .se” 域升级时的“脚本配置错误”是引起 这次网络故障的原因。 • •       很显然,程序末尾仅少了个句号使得域名系统( DNS )无法识 别“ .se ” 了,“ .se” 是瑞典的“顶级”(国家)域。(译注:“ se” 取自  Sweden ,就 像“ cn” 取自中国 china 一样) •       在对脚本测试期间,这个遗漏的句号没有被发现。而该软件一旦投入运行, 监视系统便发现该遗漏的句号,并生成一个新文件。 • •       然而,由于旧脚本信息缓存在各互联网服务提供商( ISP )中,要等到各 ISP 重新还原系统,由新脚本引起的中断才告结束。修正后的新脚本在当地时间晚上 10 时 43 分投入运行。
  • 24. Bug 25: 携程 • 点数换机票,需要上网认证,并通过 手机认证,客户收到认证码后,再上 网确认。 • 里程部看到的是里程数已经可以用了 • 而客服部看到的是还不能够购买 • 客户在这两个部门间被踢来踢去,一 个很好的客户满意计划变成了客户抱 怨计划。
  • 25. Bug 26: 意大利邮电局系统更新 Bug• 2009 年 11 月 25 日邮局系统更新,包 括所有的 ATM • 整数后面的小数点被去掉,取 115.00 欧元被认为是取了 11500 欧元 • 上万人的账户显示透支,不能再使用 。 • 客服电话被打爆
  • 26. Bug 27: 微软 Office 2003 权限 Bug• Cannot Open Office 2003 Documents Protected with RMS • Starting on December 11, 2009, customers using Office 2003 will not be able to open Office 2003 documents protected with the Rights Management Service (RMS) or save Office 2003 documents protected with RMS. The following error message may be displayed when attempting to Open RMS Documents using Office 2003:
  • 27. Bug 28: 微软手机 Bug • Messages received after 1/1/2010 may be dated as 2016 • Today's date 010110 • BCD 10 is 0001 0000 in binary, which is 16 in decimal. • Bank of Queensland’s Eftpos terminals. OQ’s Eftpos machines skipped ahead six years when the clock ticked over to January 1 and started date stamping January 2016.
  • 28. Bug 29: SpamAssassin Bug • Messages received after 1/1/2010 are all treated as Spam • Promptly at the start of the new year, all mails started getting an extra 3.4 points based on FH_DATE_PAST_20XX: header FH_DATE_PAST_20XX Date =~ /20[1-9][0-9]/ [if-unset: 2006] describe FH_DATE_PAST_20XX The date is grossly in the future.
  • 29. Bug 30: Mars Pathfinder Bug • 2+2 = 5 check • 一个产生偶数的算法 • 实验室中只发生了一次,无法重现 • 中断发生了一次,在执行算法之前(百 万分之一秒)