Modern FreeBSD (at Opentech 2010)
•Als KEY, PDF herunterladen•
0 gefällt mir•951 views
Outline of modern elements in FreeBSD 8. See Section 4c of http://www.ukuug.org/events/opentech2010/schedule for audio for the session. This presentation starts around the 20 minute mark.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Empfohlen
Empfohlen (20)
Modern FreeBSD (at Opentech 2010)
- 1. Modern FreeBSD Industrial Strength UNIX Words: Robert Watson, University of Cambridge, rwatson@FreeBSD.org Editing, Presentation: Mark Blackman, mark@exonetric.com
- 2. What is FreeBSD - Technical Open source BSD UNIX-derived OS ISP server network server platform Yahoo!, Verio, NY Internet, ISC, ... Appliance/product/embedded OS foundation Juniper JunOS, Nokia, Panasas, Timing Solutions,... VXWorks, Mac OS X, Netapp, Apple iOS FreeBSD components in Solaris, BT Home hub. One of the most successful open source projects Focus on storage, networking, security
- 3. Unix_history-simple.svg Pedigree 10/09/2010 22:28 1969 Unics 1969 Open Source 1971 to 1973 UnixTSS 1 to 4 1971 to 1973 Mixed/Shared Source 1974 to 1975 UnixTSS 5 to 6 PWB/Unix 1974 to 1975 Closed Source 1978 BSD 1978 1.0 to 2.0 UnixTSS 1979 7 1979 Unix 32v 1980 BSD 1980 3.0 to 4.1 Xenix 1981 1.0 to 2.3 System III 1981 1982 Xenix 1982 BSD 4.2 3.0 1983 Sun OS System V 1983 1 to 1.1 R1 to R2 1984 SCO Xenix 1984 UnixTSS 1985 8 SCO Xenix AIX System V 1985 W286 1986 BSD 4.3 1.0 R3 HP/UX Sun OS 1.0 to 1.2 1986 1.2 to 3.0 SCO Xenix 1987 UnixTSS V386 1987 (Time Sharing HP/UX 1988 System) BSD 4.3 System V R4 2.0 to 3.0 1988 9 to 10 Tahoe SCO Xenix 1989 W386 1989 BSD 4.3 1990 Reno 1990 1991 BSD NET/2 Linux 0.0.1 1991 Sun OS Minix 4 1.x NEXTSTEP/ 386BSD OPENSTEP 1992 1992 1.0 to 4.0 HP/UX NetBSD 6 to 11 Linux 0.8 to 1.0 1993 0.95 to 1.2.x BSD SCO Unix 1993 4.4 to Unixware 3.2.4 1994 FreeBSD 4.4 lite2 1.x to 2.x 1994 1.0 to 1995 2.2.x NetBSD OpenBSD OpenServer 1995 1.1 to 1.2 1.0 to 2.2 Solaris 1996 AIX 5.0 to 5.04 2.1 to 10 1996 3.x to 6.x 1997 1997 NetBSD 1.3 1998 FreeBSD 1998 3.0 to 3.2 Minix OpenServer Unixware 1999 2.x Mac OS X 5.0.5 to 5.0.7 7.x Linux Server 2000 1999 2.0 to 2.6.x OpenBSD 2.3 to 4.x 2000 FreeBSD NetBSD 2001 to 2004 3.3 to 8.0 1.3 to 5.x 2001 to 2004 Mac OS X HP/UX 2005 10.0 to 10.6 11i to 11i v3 OpenServer OpenSolaris 2005 Minix (Darwin) 3.x 6.x 2008.05 and 2006 to 2010 later 2006 to 2010 http://commons.wikimedia.org/wiki/File:Unix_history-simple.svg
- 4. What is FreeBSD - Social Global community of developers and users FreeBSD.org web site, mailing lists, forums Academic research platform: Cambridge, Berkeley, Rice and others Developer community: Core team, Committers, Ports maintainers, Contributors User communities Some more or less involved in global community Companies: find the BSD licence less demanding than GPL and contribute back.
- 5. Modern Pressures Multiprocessing now dominant paradigm Convergence of hardware platforms Granular security model requirements Storage management The evolving role and nature of open source
- 6. Threading In the mid-1990's, it was unclear what concurrent programming model would win. maturing thread models in FreeBSD over the last decade pthreads project in the mid-1990's followed by KSE, an M:N threading model most recently, libthr. Application models evolved, and simpler and less concurrent 1:1 implementation offered greater benefit
- 7. Threading FreeBSD engaged with Apple to adapt Grand Central Dispatch (GCD) builds on threads and the FreeBSD kqueue model offers a more programmer-accessible, scalable, programming model
- 8. Tracking Processor Changes transition from instruction-centric to cache-centric performance support for newer and widespread RISC architectures (ARM, MIPS, PowerPC, Sparc64) Embedded systems with virtual memory support makes FreeBSD an ideal fit
- 9. Parallel Hardware Ten years ago, FreeBSD ran on two-core systems, FreeBSD runs on 32-core systems, with 64,128-core configurations in our immediate future Giant locked kernel to granular, data-oriented locking (the very successful 10-year SMPng project). Continous R&D on algorithms and approaches for high core counts.
- 10. Parallel Hardware FreeBSD's performance in multiprocessing is cutting edge. Now developing a highly scalable network stack able to run on dozens(perhaps someday hundreds) of kernel threads. Collaborating with network vendors such as BlueCoat and Juniper hardware vendors such as Chelsio and Intel. Likewise, NUMA is becoming a reality.
- 11. Virtualization FreeBSD has lead and responded to trends FreeBSD's Jail model, introduced around ten years ago, led a revolution in OS-level virtualization, with imitators such as Solaris Zones Linux Virtual Servers ( OpenVZ/Virtuozzo, Linux- VServer, FreeVPS) Runs on Xen, Virtualbox as well.
- 12. Security models UNIX model was powerful, but limiting TrustedBSD Project, announced roughly ten years ago. Our MAC Framework allows flexible and extensible access control. Funded by DARPA, US Navy, Apple, McAfee and others. extend the OS model without modifying the OS. ie. sandboxing in iPhone OS apps, JunOS routers. Relied on throughout military, industry, academia
- 13. Security Models Companies like Apple, Juniper, and McAfee use MAC framework extensively to provide alternative access control models, such as sandboxing, for third-party code, build system assurance. Capsicum: “capability” oriented security, out of Cambridge, collaboration with Google, possible ChromeOS component.
- 14. Storage advancing the UFS file system over the last decade GEOM framework, developed under sponsorship from DARPA in the early 2000's ported Sun's ZFS file system to FreeBSD
- 15. Development/Introspection new clang/LLVM compiler suite Coverity's Prevent static analysis tool FreeBSD has ported Sun's DTrace userspace Dtrace support coming. hardware performance monitoring counters support Kernel debugging tools
- 16. Spinoffs Customize FreeBSD's powerful OS core The open source side has greatly expanded our user reach as an open source system. PC-BSD, a desktop-oriented system. pfSense, a firewall system. FreeNAS, a network attached storage system. FreeBSD integrates back core OS features.
- 17. Open Source Project Management Project growth FreeBSD's social organization Elected core team model Huge OS development projects, c.f. SMPng Delegated project leadership structure No more “core team” CVS to Subversion. Perforce
- 18. Summary While the outside looks much the same, under the hood FreeBSD is a modern OS by any definition. Commercial cooperation at all levels has enhanced FreeBSD beyond what an “open source only” approach permits. Cutting edge research. The biggest open source project you’ve never heard of.
Hinweis der Redaktion
- - embedded now uses similar technologies to servers - SMP dominance - (which motivates SMPng, GCD, ...) - ZFS and UFS improvements - we've lead the way in identifying how open source projects should run, and what they should do, but have done a lot of experimentation to get there
- 1:1 - simplifying scheduler data structures and allowing them to use more complex heuristics
- GCD - a new concurrent programming framework, answers mapping of M:N to 1:1 we switched from M:N to 1:1 to simply the scheduler / threading code, and because it appeared that application writers were generally choosing to use small thread pools (say, 2-100 threads) rather than using very large numbers. In part because Linux used 1:1 and you couldn't get it to create very large thread counts
- Apple’s FreeBSD-derived iOS on iPhone and Ipad FreeBSD-derived JunOS running on low-power switches
- Recent work on algorithms and approaches that scale to high core counts through complex work distribution, and hardware-assisted work distribution on high-performance network cards
- 1. as benchmarks over the last few years have shown, and we continue to aggressively exploit new parallelism.   2. and our kernel memory allocator has recently been updated to introduce NUMA-awareness.
- 1. for many evolving applications such as firewall appliances and smartphones.
- 3. Most recently, we've announced the Capsicum project, developed in collaboration with Google, to support capability-oriented OS security. 4. capiscum cutting edge, best paper
- 64-bit support snapshots background file system checking extended attributes advanced security features such as ACLs and MAC and most recently, journaling 2. allows easily pluggable and extensible storage transforms, * storage multipathing * new RAID integration * full file-system journaling and * replication for fault tolerance.   3. with its self-healing and management features
- 1. we even have our own Prevent server system and actively re-analyze our source code nightly.   2. offering introspection tools for performance and behavioural analysis 3. allows similar analysis of userspace applications the FreeBSD foundation is sponsoring Rui Paulo to complete this. 4. make FreeBSD one of the best OS platforms for kernel feature development, including a. integrated debugging b. dynamic lock order analysis Talked to many developers who actually write their Linux kernel code on FreeBSD so that they can use the debugging tools, before porting to Linux!
- Commercial vendors have always done this: NetApp, Juniper, Apple, Isilon, Panasas, ...
- 1. Dealing with project growth has been a critical challenge, both socially and technically.   1a. A social experiment that has proven extremely successful, allowing us to renew project leadership over time. 2. Roughly ten years ago, we moved for the first time to an 3. including the very interesting SMPng project, which involved dozens of developers (and companies) collaborating to improve multiprocessor scalability. 4.  in which chartered teams take responsibility for portions of the project's work: release engineering, application porting, security advisories, system adminstration, bug-busting, documentation, etc, 5. Moving away from the idea that a small "core team" of developers does it all. 6. CVS: many extensions --> Subversion: few extensions 7. as a way to improve our branched development methodology for side-projects with long life cycles, as well as supporting dozens of summer students each year sponsored by Google 8. Our most recent developer summit at BSDCan 2010 in Ottawa had over 100 attendees including developers and invited guests from various companies.  That's a good bit bigger than the 15-20 (?) folks at the first developer summit at the 2001 USENIX ATC in Boston. 9. Warner’s audit of all involved licenses should be mentioned also.