How to master OpenStack in 2 hours

1. TRYSTACK.CN PyConChina CodeLab How to master OpenStack in 2 hours TryStack.cn Meetup #ShangHai 8th November 2013(update) @ben_duyujie Duyuje.dyj@gmail.com http://www.slideshare.net/ben_duyujie/how-to-master-openstack-in-2-hours

2. TRYSTACK.CN Who am I? : @ben_杜玉杰 www.linkedin.com /in/duyujie Website: www.du yu http://about.me/Y jie.org ujie.Du

3. TRYSTACK.CN Agenda How to set up your TryStack Lab What's OpenStack? Why we are here? Devstack in 5 min First contact with OpenStack Write a template for Heat

4. TRYSTACK.CN How many people have used OpenStack? Put up your hands!

5. TRYSTACK.CN Prepare for Labs ● VM − Vmware, VirtualBox, vagrant, KVM, …, any choice − 2GB RAM or more ● OS − Ubuntu 12.04 (Precise), Fedora 17 and CentOS/RHEL 6.4 − OpenSUSE and Debian − Minimal install recommended ● Download trystack.box

6. TRYSTACK.CN 0.Tools ● VirtualBox 4.2.20 https://www.virtualbox.org ● Vagrant v1.3.4 http://downloads.vagrantup.com RDO/Puppet/Chef Box1/VM1 Box2/VM2 Vargant ● Centos64-x86_64-20131030 VirtualBox http://www.vagrantbox.es/ Ubuntu/CentOS/MAC

7. TRYSTACK.CN 1.Install VirtualBox For Host OS, Ubuntu 12.04: $sudo echo deb http://download.virtualbox.org/virtualbox/debian precise contrib >> /etc/apt/sources.list $ wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox. asc -O- | sudo apt-key add -” $ apt-key finger | grep VirtualBox $ sudo apt-get update $ sudo apt-get install virtualbox-4.2 sudo dpkg -r virtualbox-4.2 && sudo reboot

8. TRYSTACK.CN 2.Install VirtualBox Extension Pack $sudo apt-get install vim git expect $sudo wget http://dlc.sun.com.edgesuite.net/virtualbox/4.2.20 /Oracle_VM_VirtualBox_Extension_Pack-4.2.20-90983.vbox-extpack $sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.2.20-90983.vbox-extpack

9. TRYSTACK.CN 3. Set up Vagrant $ mkdir trystack $ cd trystack $ vagrant init $ vim Vagrantfile # -*- mode: ruby -*# vi: set ft=ruby : VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "Trystack" config.vm.box_url = "Centos64-x86_64-20131030" config.vm.network :private_network, ip: " 172.16.0.2" config.vm.synced_folder "../devstack", "/vagrant_data" config.vm.provider :virtualbox do |vbox| vbox.customize ["modifyvm", :id, "--memory", "2048"] end end

10. TRYSTACK.CN 4. PIP Devstack will install EPEL and RDO repo for CentOS6.4, Set up pypi : $mkdir $HOME/.pip $vi $HOME/.pip/pip.conf [global] index-url = http://pypi.douban.com/simple/ -------------------exit, reboot to update env.

11. TRYSTACK.CN vagrant up vagrant box add trystack Centos64-x86_64-20131030.box vagrant up vagrant ssh

12. TRYSTACK.CN 常用的vagrant命令 $ vagrant box add NAME URL $ vagrant box list #添加一个box #查看本地已添加的box $ vagrant box remove NAME virtualbox #删除本地已添加的box，如若是版本1.0.x，执行$ vagrant box remove NAME $ vagrant init NAME #初始化，实质应是创建Vagrantfile文件 $ vagrant up #启动虚拟机 $ vagrant halt #关闭虚拟机 $ vagrant destroy #销毁虚拟机 $ vagrant reload #重启虚拟机 $ vagrant package #当前正在运行的VirtualBox虚拟环境打包成一个可重复使用的box $ vagrant ssh #进入虚拟环境 https://github.com/astaxie/Go-in-Action/blob/master/ebook/zh/01.0.md

13. TRYSTACK.CN 5. Install DevStack Home pages: http://devstack.org Three steps： $ git clone https://github.com/openstack-dev/devstack.git $ cd devstack $ ./stack.sh exercise.sh: setting up the env for running the sample exercise cmds exercises/ folder: contains a small set of exercise file for achieving different functionalities

14. TRYSTACK.CN Files localrc: ● localrc is a user-maintained settings file that is sourced from stackrc. This gives it the ability to override any variables set in stackrc. Samples: http://devstack. org/localrc.html stackrc: ● git repo and branch information, image download location. This file shouldn't be modified, only localrc should be modified. stack.sh: ● stack.sh is an opinionated OpenStack developer installation. It installs and configures various combinations of Glance, Horizon, Keystone, Nova, Neutron, Swift and some other projects.

15. TRYSTACK.CN Chang to Grizzly ● 可以在localrc文件中添加以下内容： − NOVA_BRANCH=stable/grizzly − CINDER_BRANCH=stable/grizzly − GLANCE_BRANCH=stable/grizzly − KEYSTONE_BRANCH=stable/grizzly − NEUTRON_BRANCH=stable/grizzly − HORIZON_BRANCH=stable/grizzly

16. TRYSTACK.CN localrc http://www.sebastien-han.fr/blog/2013/08/16/best-localrc-for-devstack/ $cd devstack echo ADMIN_PASSWORD=trystack >> localrc cho MYSQL_PASSWORD=trystack >> localrc echo RABBIT_PASSWORD=trystack >> localrc echo SERVICE_PASSWORD=trystack >> localrc echo SERVICE_TOKEN=trystack >> localrc HOST_IP=172.16.0.2 echo HOST_IP=$HOST_IP >> localrc echo SERVICE_HOST=$HOST_IP >> localrc echo MYSQL_HOST=$HOST_IP >> localrc echo RABBIT_HOST=$HOST_IP >> localrc echo Q_HOST=$HOST_IP >> localrc echo GLANCE_HOST=$HOST_IP >> localrc echo G_HOST=$HOST_IP >> localrc echo GLANCE_HOSTPORT=$HOST_IP:9292 >> localrc echo HOST_IP_IFACE=eth1 >> localrc echo FLAT_INTERFACE=br100 >> localrc echo PUBLIC_INTERFACE=eth1 >> localrc echo FLOATING_RANGE=172.16.0.224/27 >> localrc

17. TRYSTACK.CN Continue # Enables all the Heat services echo ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng >> localrc ## It would also be useful to automatically download and register VM images that Heat can launch. # 64bit image (~660MB) echo IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-x86_64-cfntools.qcow2" >> localrc echo IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F18-x86_64-cfntools.qcow2" >> localrc # 32bit image (~640MB) echo IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-i386-cfntools.qcow2" >> localrc echo IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F18-i386-cfntools.qcow2" >> localrc source localrc echo VORBOSE=True >> stackrc echo LOGFILE=stacklog.txt >> stackrc source stackrc

18. TRYSTACK.CN Wait … ./stack.sh # Devstack is now installed. # If all is well, you will see the following output : # Horizon is now available at http://IP_ADDRESS/ # Heat has replaced the default flavors. View by running: nova flavor-list # Keystone is serving at http://IP_ADDRESS:5000/v2.0/ # Examples on using novaclient command line is in exercise.sh # The default users are: admin and demo # The password: trystack # This is your host ip: IP_ADDRESS # stack.sh completed in 1065 seconds.

19. TRYSTACK.CN Users 默认Devstack创建 admin和demo两个用户，通过设置环境变量可以进行切换： admin: source openrc admin admin demo: source openrc demo demo 代码目录：/opt/stack可以设置为开发工具工作目录 $ screen -x stack 参考：http://www.ibm.com/developerworks/cn/linux/l-cn-screen/

20. TRYSTACK.CN OpenStack 经验值 +10

21. TRYSTACK.CN Let's begin to introduce Openstack What's OpenStack?

22. TRYSTACK.CN

23. TRYSTACK.CN Linux Kernel Similarities ● ● ● ● ● Open Source Released early OpenStack Foundation is very similar to the Linux Foundation Replaced the proprietary incumbents One could consider the abstractions quite similar too:

24. TRYSTACK.CN Cloud Operating System

25. TRYSTACK.CN Havana 如何做到的 OpenStack由一系列的子项目组成，形成一个完整的 IaaS解决方案，？

26. TRYSTACK.CN

27. TRYSTACK.CN 持续集成

28. TRYSTACK.CN Development Development done on always open trunk Releases done to a stable branch Standardized tools across all projects − Git − Gerrit ● Standalone patch review server ● Created by Google for Android development − Jenkins ● Continuous integration ● Gates commits on unit tests

29. TRYSTACK.CN Commit process ● git branch; git commit − Commit hook inserts Change-Id: for gerrit tracking ● ./run_tests.sh [subset] − Unit tests within a virtualenv − Nose used to extend unittest − Pep8 enforced https://github.com/jcrocholl/pep8 ● git-review python tool used to submit to gerrit − Developed for OpenStack but not specific to it

30. TRYSTACK.CN Review Process Gated trunk − Reviewers ● Anyone can ±1 ● Only project core members can approve − Automated tests ● Through Jenkins ● Using OpenStack infrastructure itself ● Sets up test systems and runs gerrit branch ● − Libvirt, Xen, python 2.6, python 2.7, ... If OK, merges gerrit branch to trunk ● Ancillary jobs like doc builds, pylint reports, more tests, ...

31. TRYSTACK.CN OpenStack 发布周期: 四个阶段 Design* Development QA Release

32. TRYSTACK.CN Release History ● 6 monthly, design summit after each, monthly milestones ● Austin Oct 2010 Nova + Swift ● Cactus Apr 2011 Glance ● Diablo Sep 2011 First “production ready” release ● Essex Apr 2012 Horizon + Keystone ● Folsom Sep 2012 Quantum + Cinder ● Grizzly Apr 2013 Neutron ● Havana Oct 2013 Heat +Ceilometer

33. TRYSTACK.CN Openstack is the community • Interact via: – Mailing lists: general, development, documentation, operators … – Ask Openstack (ask.openstack.org) – Launchpad ■ Home to all openstack integrated projects – IRC (#openstack-101, #openstack, #openstack-dev, …) – Local User Groups (like today!) – Summit & Conference (twice a year)

34. TRYSTACK.CN See Openstack evolving, everyday • http://status.openstack.org – Release status – Active Reviews • https://github.com/openstack – Grab the code

35. TRYSTACK.CN Why we are here?

36. TRYSTACK.CN

37. TRYSTACK.CN OpenStack Operational details of a large python project

38. TRYSTACK.CN Python technologies used (Havana)

39. TRYSTACK.CN Who develops OpenStack $ cd /opt/stack/keystone $ git log | grep Author | 17 99cloud.net 18 ca.ibm.com 18 canonical.com 19 metacloud.com 20 suse.cz sed -n 's/.*<.*@(.*)>/1/p' | 33 linux.vnet.ibm.com 36 chmouel.com 36 hp.com 37 inaugust.com 46 openstack.org sort | uniq -c | sort -n | 55 hcl.com 62 mac.com 91 us.ibm.com 171 yahoo.com 232 redhat.com tail -n20 | column 276 anarkystic.com 306 highbridgellc.com 421 rackspace.com 727 gmail.com 1007 review.openstack.org

40. TRYSTACK.CN Related python projects ● OZ − Generate guest images from install ISOs ● Optional customization − − Install packages, run script, ... List of OZ built images from RackSpace ● https://github.com/rackerjoe/oz-image-build − ISO -> image -> glance -> nova -> libvirt -> KVM − http://aeolusproject.org/oz.html

41. TRYSTACK.CN Related python projects ● Heat API − AWS Cloudformation type service − V6 just released − Provisions apps in the cloud − And keeps them provisioned ● − Leverages cloud attributes to provide HA https://github.com/heat-api

42. TRYSTACK.CN Related python projects ● cloud-init − Installed in guests to configure at boot time − Directives for package install etc. − Support for many distros − https://launchpad.net/cloud-init/

43. TRYSTACK.CN Ryu Plugin for Neutron Ryu is a component-based software defined networking framework. Ryu provides software components with well defined API that make it easy for developers to create new network management and control applications. Ryu supports various protocols for managing network devices, such as OpenFlow, Netconf, OF-config, etc. About OpenFlow, Ryu supports fully 1.0, 1.2, 1.3 and Nicira Extensions. Home Pages:http://osrg.github.io/ryu/

44. TRYSTACK.CN Similar non python projects ● oVirt - java − More aligned with private datacenters ● Eucalyptus - C − Less general, Closed editions. ● CloudStack - java − Started closed (cloud.com). Closed addons. ● OpenNebula - C++ − Again focusing on datacenters

45. TRYSTACK.CN Trying it out ● http://trystack.org/ Or http://cloud.trystack.cn − ● http://devstack.org/ − ● Register to try out a test public OpenStack cloud Script to setup simple/developer installs fedoraproject.org/wiki/Getting_started_with_OpenStack − Basic setup instructions for machine or VM

46. TRYSTACK.CN What's trystack.cn ● Biggest OpenStack testbed in China − − Opened OpenStack API − ● Non-profit project Provide best practices for administrator, operators and developers. Reference Architecture − − OpenStack deployment architecutre − ● Hardware devices from different vendors Using the newest OpenStack code Contribute code to the community − − ● Bug fixes New features Promote OpenStack implementation and productization in China With partners, vendors

47. TRYSTACK.CN More info ● http://docs.openstack.org/ − The compute admin guide is a good overview ● http://ci.openstack.org/ − OpenStack Continuous Integration info ● http://review.openstack.org/ − OpenStack gerrit interface ● http://jenkins.openstack.org/ − OpenStack Continuous Integration interface

49. TRYSTACK.CN Are you ready?

50. TRYSTACK.CN Login cloud.trystakc.cn OpenStack Havana Demo Havana Dashboard

51. TRYSTACK.CN Help $ keystone help $ keystone help user-create $ glance help $ glance help image-create $ nova help $ nova help boot

52. TRYSTACK.CN ● 管理网络 DevStack单节点部署Neutron时需要在localrc里设置： − disable_service n-net − enable_service q-svc − enable_service q-agt − enable_service q-dhcp − enable_service q-l3 − enable_service q-meta − enable_service neutron − # Optional, to enable tempest configuration as part of devstack − enable_service tempest ● 重新安装DevStack 参考：http://devstack.org/lib/neutron.html

54. TRYSTACK.CN

55. TRYSTACK.CN Heat Overview Horizon Dashboard Heat Orchestration Nova Compute Node Keystone Glance Swift Quantum Image Service Object Store Networkin g Cinder Volume Service ● Provides AWS Cloudformation and native ReST API ● Abstract configuration of services to single-template ● HA/Autoscaling/Monitoring features ● Openstack incubated project Identity Service

56. TRYSTACK.CN Heat Overview ● Orchestration of Openstack deployments ● Integrates with all Openstack core projects ● Converts a JSON template into a cloud application ● Implements well known template and API (AWS Cloudformation, also YAML, ReST) ● Version your cloud applications like your software ● Repeatable deployments, fully automated

57. TRYSTACK.CN Heat Overview

58. TRYSTACK.CN Heat Stack lifecycle ● Base OS image stored in glance ● Deploy instances & configure services based on template contents (resources) ● Deployment-time configuration/customization ● Very flexible, but much reduced complexity ● Cloud-init (nova user/metadata), plus cfn-init ● Puppet/Chef can still be used if you want! ● Fully integrated single-service solution

59. TRYSTACK.CN Heat Stack lifecycle Stack Template RES RE ST RE ST T ES R Nova T Heat-jeos (oz) Glance (Image Store) ST T RERES ST RE Heat Nova-network /quantum Nova-volume /cinder Instance - cloud-init - cfn-init - cfn-hup -cfn-push-stats Swift Ceilometer

60. TRYSTACK.CN Heat Stack lifecycle

61. TRYSTACK.CN Why Heat/orchestration? ● Orchestration makes things repeatable ● Much easier to provide “on demand” infrastructure ● Much less scripting, single template system ● Leverage the power of openstack without learning fine detail of every service (learn one set of APIs/tools) ● Portability/migration ● Version/tag/branch your infrastructure like code! ● Composed templates/modularity ● Pluggable, user-modifiable resource implementations ● Open(stack) community project :)

62. TRYSTACK.CN Definition: Template Parameters "Parameters" : { "InstanceType" : { "Description" : "WebServer EC2 instance type", "Type" : "String", "Default" : "m1.large", "AllowedValues" : [ "t1.micro", "m1.small", "m1.large", "m1.xlarge", "m2.xlarge","m2.2xlarge", "m2.4xlarge", "c1.medium", "c1.xlarge", "cc1.4xlarge" ], "ConstraintDescription" : "must be a valid instance type." }, } Using the Parameter: { “Ref” : “InstanceType” }

63. TRYSTACK.CN Definition: Template Mappings “Mappings”: { "DistroArch2Inst": { "F16" : { "32" : "F16-i386-cfntools", "64" : "F16-x86_64-cfntools" }, "F17" : { "32" : "F17-i386-cfntools", "64" : "F17-x86_64-cfntools" }, "U10" : { "32" : "U10-i386-cfntools", "64" : "U10-x86_64-cfntools" } } } Using the Mapping: “ImageId”: { "Fn::FindInMap" : [ "DistroArch2Inst", { "Ref" : "Distribution" }, { “Ref” : “Arch” } ] }

64. TRYSTACK.CN Template Resources Resources { “WikiDatabase”: { "Type" : "AWS::EC2::Instance", .. bunch of stuff ... }, "DatabaseIPAddress" : { "Type" : "AWS::EC2::EIP" }, "DatabaseIPAssoc" : { "Type" : "AWS::EC2::EIPAssociation", "Properties" : { "InstanceId" : { "Ref" : "WikiDatabase" }, "EIP" : { "Ref" : "DatabaseIPAddress" } } } }

65. TRYSTACK.CN Template Resources (YAML) Resources: WebServer: Type: AWS::EC2::Instance ..bunch of stuff.. IPAddress: {Type: 'AWS::EC2::EIP'} IPAssoc: Type: AWS::EC2::EIPAssociation Properties: InstanceId: {Ref: WebServer} EIP: {Ref: IPAddress}

66. TRYSTACK.CN Heat Resource Types

67. TRYSTACK.CN In Closing ● Users and developers wanted! − − Check out the repository: − ● Connect with the community via IRC on #heat@freenode https://github.com/openstack/heat Read the Documentation: http://wiki.openstack.org/Heat/ Heat simple but powerful method for orchestrating OpenStack environments

69. TRYSTACK.CN Let's try it !

70. TRYSTACK.CN Heat ● $ heat stack-create trystack -f WordPress_Single_Instance.template -P "InstanceType=m1.small;DBUsername=wp;DBPassword=trystack; KeyName=demo_heat_key;LinuxDistribution=F17" ● $heat stack-list ● $nova list ● $heat stack-show trystack 查看网站URL ● $wget URL

71. TRYSTACK.CN CLI add keypair： $ nova keypair-add trystackkey > trystack.pem $ nova keypair-list

72. TRYSTACK.CN localrc ADMIN_PASSWORD=trystack MYSQL_PASSWORD=trystack RABBIT_PASSWORD=trystack SERVICE_PASSWORD=trystack SERVICE_TOKEN=trystack # Optionally alter HEAT_REPO to use a fork. HEAT_REPO=https://github.com/sjcorbett/heat.git HEAT_BRANCH=master # Enable Heat ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng IMAGE_URLS+=",http://fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-i386-cfntools.qcow2,http: //fedorapeople.org/groups/heat/prebuilt-jeos-images/F17-x86_64-cfntools.qcow2"

73. TRYSTACK.CN ● ● cp /vagrant/F17-i386-cfntools.qcow2 ~/devstack/files/ $ ./stack.sh

74. TRYSTACK.CN ● $ source openrc ● $ nova flavor-list

75. TRYSTACK.CN Set Security Group $ nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0 $ nova secgroup-add-rule default tcp 22 22 0.0.0.0/0 $ nova secgroup-add-rule default tcp 80 80 0.0.0.0/0 $ nova secgroup-list-rules default +-------------+-----------+---------+-----------+--------------+ | IP Protocol | From Port | To Port | IP Range | Source Group | +-------------+-----------+---------+-----------+--------------+ | icmp | -1 | -1 | 0.0.0.0/0 | | | tcp | 22 | 22 | 0.0.0.0/0 | | | tcp | 80 | 80 | 0.0.0.0/0 | | +-------------+-----------+---------+-----------+--------------+

76. TRYSTACK.CN

77. TRYSTACK.CN

78. TRYSTACK.CN

79. TRYSTACK.CN

80. TRYSTACK.CN

81. TRYSTACK.CN

82. TRYSTACK.CN Create a stack

83. TRYSTACK.CN $ heat stack-show trystack 注意：这里stack_status里的CREATE_COMPLETE 只是表示OpenStack资源已被创建，但实时上并没有安装完WordPress及MySQL等，估计需要十几分钟

85. TRYSTACK.CN 你的经验值是多少？

86. TRYSTACK.CN Thanks! About - http://about.me/Yujie.Du Email – du.yujie@trystack.cn Twitter - @ben_duyujie Blog - http://www.duyuje.org G+ - https://plus.google.com/+YujieDu LinkedIn - http://www.linkedin.com/in/duyujie SlideShare -http://www.slideshare.net/ben_dyujie

87. TRYSTACK.CN TryStack训练营我们的TryStack训练营提供一整套深入浅出了解云计算架构和设计的培训方案。如果您想了解OpenStack™的所有组件是如何协同或独立工作的，或者想评估OpenStack™是否适用于您的生产环境，那么该训练营将为您量身打造一个循序渐进的学习计划。 TryStack训练营是一套为期四-五天的针对系统管理员和架构师的培训课程，基于最新的 OpenStack版本（目前是使用的2013年10月份刚刚发布的Havana版本）。在最开始的两天，我们将从部署OpenStack环境开始带领各位逐个学习 OpenStack™的相关组件，了解并熟悉各个组件的相关特性，从整体架构层面来理解它们是如何协同工作的。在随后的两天，我们还将分享和讨论多种可行的存储和网络架构，当然，还有Openstack的一些高可用(HA)特性。在TryStack训练营中，我们还将学习如何根据你的业务需求提供相应的基于 OpenStack™的云计算的解决方案。

88. TRYSTACK.CN TRASTACK Camp for DELL

89. TRYSTACK.CN TRASTACK Camp for HP

90. TRYSTACK.CN 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. TRYSTACK MEETUPS Oct 2012, Grizzly Summit www.slideshare.net/openstack/trystack-introfinalpdf Dec 2012, ECUG #ChengDu http://www.ecug.org/ May 2013, open souce convention #Beijing http://bj-opensource.com/ February 2013 Trystack 1st meetup #Nanjing www.openstack. org/blog/2013/02/trystack-cn-2013-first-meetup/ February 2013, Trystack 2nd meetup #Shanghai http://www.openstack. org/blog/2013/02/trystack-2nd-meetup-2013/ June 2013, Trystack 3rd meetup #Beijing in IBM http://www.openstack. org/blog/2013/06/trystack-cn-meetup-with-ibm-in-beijing/ June 2013, Trystack 4th meetup #Shenzhen in Huawei June 2013, 8th OpenSource China Open World Forum(OCOW) #Beijing http://ocow. csdn.net/ OpenStack's 3rd Birthday Celebration http://trystack.eventbrite.com/ OpenStack Track http://www.cloudconnectevent.cn/ Pycon China http://cn.pycon.org/2013

How to master OpenStack in 2 hours

Du liest eine Vorschau.

Hier ansehen

How to master OpenStack in 2 hours

Weitere Verwandte Inhalte

Diashows für Sie (20)

Andere mochten auch (20)

Ähnlich wie How to master OpenStack in 2 hours (20)

Weitere von OpenCity Community (20)

Aktuellste (20)