REVENGE Posted October 26, 2008 Share Posted October 26, 2008 Hi guys, welcome to the OS X Live project team. Before this new development cycle begins, I would like you all to get to know each other a little bit. Feel free to talk about your skills and interests, any relevant or related projects you've have worked / are working on, and exchange contact information with each other as needed (do note that this thread is currently public, and can be viewed by all browsers, so if you want to keep your contact info private, please exchange info using PMs). Following that, I would suggest you guys begin by laying out a development plan / roadmap. Personally, I give the following recommendations: 1. Familiarize yourselves with what the previous developers have already achieved. You can check out relevant information by visiting the Archives section of this subforum. 2. Understand what's needed to get the latest release of OS X working in a live environment (DVD or USB removable drive), taking note of the following requirements (note that these specification I'm making aren't necessarily designed for a user customized Live system, but rather for a generic one akin to say the main Knoppix or Slax distro): a. Either run the userland using RAM drive(s) or you construct a nice low-level read/write filter to protect the lifespan of flash memory devices. b. Try to include as many system/device drivers as possible for different system configurations (Intel/AMD, Intel/NVIDIA/AMD(ATi), laptop/desktop, wired/WiFi, etc.). Keep in mind that some (or many) of these driver packages may not be compatible with / may interfere with each other, so the best scenario would be to have either a way for the user to manually configure the hardware settings or (preferably) have an auto-detection system/script that would do so automatically. c. Do NOT sacrifice portability for speed/optimization. For example, I'd rather see one distro with an AMD/Intel compatible kernel than several distros with Intel/AMD/SSE2/SSE3/EFI kernels. If you can put them on one disk, great! If not, make sure you include the MOST compatible kernel you can find. 3. In addition to achieving a nice core system setup, work on improving the user experience. It has already been demonstrated that the full OS X default shell (Dock, Finder, etc.) can be run off of a LiveDVD, so this should be your minimum target (although, if you have a good argument for using a more minimalistic shell, it's probably fine as well). Then, in addition to this, figure out what additional niceties you should include with a Live distribution. Notice I'm starting to reference the Live Linux distros again, so an example feature that come to mind is a way for the user configuration/files to be stored to rewritable media and later retrieved. Look at what's been done for Live Linux distros, and try to do the same for OS X as well. 4. This forth point is very important: create a graphical tool/utility for generating and customizing Live DVDs. The main configuration can be something akin to Knoppix, but give the user the option of creating their own customized Live environment. There will likely be a demand for a small and efficient GUI rescue disk type systems. I would personally like to see the ability to generate either USB installs or DVD images based on profiles, so you could have a default "demo disk" profile for showing off OS X on PCs and a "rescue disk" profile for system recovery purposes. Give users the power to customize, and you'll have a winner. 5. Although your focus should be OSx86 hardware configurations, keep in mind that there is a significant demand for a portable OS X environment from the users of legit Apple hardware as well. Once again, think about those who've already tried (and succeeded in some cases, we do have a few examples around the net). 6. LEARN TO WORK TOGETHER AND DIVIDE/ASSIGN TASKS! You'll make much more progress much more quickly if you all work on specific things suited towards your individual strengths. Don't hesitate to ask each other for help, and make sure to communicate clearly and professionally with each other! If you have any forum administration related problems, please contact me or any of the other InsanelyMac admins. Everyone who's been accepted as a developer should receive moderation priveleges to this forum as well as an X Labs Dev badge within a few days. If you have important news you wish to announce to the community (release announcement, beta program, etc.), please contact me and I'll be able to publish it on FPN for you. Thanks, and I wish you good luck and successful devving! Link to comment Share on other sites More sharing options...
REVENGE Posted October 27, 2008 Author Share Posted October 27, 2008 These are some of Rammjet's suggestions that he posted in the Reader News thread, which I'm reposting here for reference. Having worked on this before, my suggestion is to concentrate on running OSX from a USB flash drive. I have seen some 16 GB drives selling for as low as $8 on sale. The DVD is very slow. It cannot be easily updated. Being read-only, it cannot store data/documents that you create or that you download. It requires RAM disks be setup to temporarily store the files that OSX creates in order to run. The flash drive is slow, but faster than the DVD. Since it is seen as a removable hard drive, it is very easily updated. It is read/write and can store data/documents that you create or that you download. It can store the files created by OSX in order to run. Since a flash drive does have limits to the number of times that it can be written, it would be advisable to turn off journalling and to implement RAM disks for those OSX temp files and for the vm files (hmmm...virtual memory in real memory - what a concept ). Typically, we customize OSX to run on the particular PC hardware we own. The previous Live DVD's were based on the modified version of OSX that we installed on our own hardware. This means that the resulting Live DVD probably would not work without modification for other PC's - such as a PC at a library or an internet cafe. But it is difficult to modify the Live DVD to work on other machines or to prepare it to work on a large variety of machines for which you do not know the configuration. Unfortunately, the flash drive method suffers the same problem. But solutions can be implemented more easily when using the read/write flash drive. If someone pursues any of this, I have one suggestion. Build upon the latest Boot-132 method which uses some external files (kernel, kexts, mkexts, etc) to boot a compressed disk image (DMG). modbin and I had already done some work in this area nearly a year ago. The existing Darwin bootloader can already handle this because it has features to support network booting. Network booting is essentially booting a compressed disk image located on a server somewhere on the network. Booting the compressed disk image is much faster. Linux Live DVD's use a compressed file system called Cloop. If you read the Wikipedia article on Cloop you find that the Apple DMG format is compared to Cloop. I would also suggest reading Amit Singh's book for info about netbooting. We successfully booted an OSx86 install DVD in 25-50 seconds using the compressed image. We did have trouble booting an actual compressed image made from OSX which had already been installed. Best of luck. Link to comment Share on other sites More sharing options...
Adam.co.jp Posted October 27, 2008 Share Posted October 27, 2008 Hello there just thought I would say hello to all of you I'm glad I got on this project because this has the prospect of becoming something HUGE for the community if when we complete it. I'm going to assume I probably have the least knowledge out of all of you, but I do learn super quick and won't be a burden that I promise. I've just been reading through the 10.4.1/4/5 Live and Install DVD post and my god modbin has done a lot. I'm going to read some more and digest all he's said. Anyway hope to talk to all the team soon. Link to comment Share on other sites More sharing options...
nightwalker12 Posted October 27, 2008 Share Posted October 27, 2008 Hello there just thought I would say hello to all of you I'm glad I got on this project because this has the prospect of becoming something HUGE for the community if when we complete it. I'm going to assume I probably have the least knowledge out of all of you, but I do learn super quick and won't be a burden that I promise. I've just been reading through the 10.4.1/4/5 Live and Install DVD post and my god modbin has done a lot. I'm going to read some more and digest all he's said. Anyway hope to talk to all the team soon. hello adam my name is john im one of the members of the team so your from london well guess what i was born in plymouth long time ago guess once everyone has got together we can see witch way and strenths we all have to make this work and witch leopard we shall start working on what areas have u worked in Link to comment Share on other sites More sharing options...
Adam.co.jp Posted October 27, 2008 Share Posted October 27, 2008 Ah I was actually born in France but just study here in London. I've been to Plymouth once or twice on a night out when I was training at Devonport I cant say it was the best of nights lol I've played around with making vista, xp and a few linux distros usb bootable. I just managed to get tiger to boot live today after playing around for a while with modbins instructions. Though I still need to try Leopard since that's what the actual project is about. I've set up quite a few rigs with Boot-132 so I've got a good idea on how to use it, which will probably useful if we use Boot-132 to get the LiveDVD working. For now we'll just have to wait for the other 3 then we can get brainstorming. Link to comment Share on other sites More sharing options...
(MoC) Posted October 28, 2008 Share Posted October 28, 2008 Hello, it's MoC, glad to be on the team. I'm from New York, and, I'll be here to help with whatever you need. Link to comment Share on other sites More sharing options...
spartango Posted October 28, 2008 Share Posted October 28, 2008 Hi, i'm spartango(lol, sn)... i've been hackintoshing(a word?) for a while now and have gotten a decent feel for the OS's internals...studying them more carefully now so we can get something done.. I've done a ton of new work on bootable Solaris and Linux/unix, and have some interesting ideas for osx based on that experience. Dunno...we'll have to see. I'm a student in the San Francisco area atm...so iguess i'll be on odd hours from 2 of you, but we'll be in touch. Team's important. BTW i think irc is our friend in terms of getting lots of stuff done..anyone? Link to comment Share on other sites More sharing options...
(MoC) Posted October 28, 2008 Share Posted October 28, 2008 @spartango: Yeah, I think they'll provide us with an IRC channel on moofspeak or if not we can just make an EFnet channel and call it a day. Link to comment Share on other sites More sharing options...
Adam.co.jp Posted October 28, 2008 Share Posted October 28, 2008 Good to meet you guys. Time zones won't be a problem with me as I practically don't sleep nowadays. Ok so now we're just missing one person. Yeah we can just set up a channel somewhere, shouldn't be a problem. Link to comment Share on other sites More sharing options...
(MoC) Posted October 28, 2008 Share Posted October 28, 2008 Good to meet you guys. Time zones won't be a problem with me as I practically don't sleep nowadays. Amen to that, now, let's get started on this. Rammjet's idea is the best thing ever, because bootable flash drives are easy to manipulate but we're also faced with drivers and compatibility. The way these wide varieties of distributions work on many computers because we're prompted to select our own drivers and applications. We can't install all the drivers/apps because they'll start to conflict with eachother and whatnot. How can we fix this? Well, this reminds me of Linux LiveCDs. They detect your hardware and load the proper drivers for your hardware. Perfect idea isn't it? OS X basically loads all of the kexts it has because there's technically no need for a detection system if you're true to Apple and running a Mac. We're lucky we even have OSx86. So, why don't we build a detection system? Tell the kernel to load our little script, which detects what you have via LSPCI or some other method, and then loads the appropriate kext. Sounds like Virtual Nirvana to me! What do you guys think? ~MoC Link to comment Share on other sites More sharing options...
Adam.co.jp Posted October 28, 2008 Share Posted October 28, 2008 Mac OS X System Startup: * Power is turned on. * Open Firmware code is executed. * Hardware information is collected and hardware is initialized. (I'll try and look up what exactly happens here. If we can modify this slightly it could be the basis of our script.) * Something (usually the OS, but also things like the Apple Hardware Test, etc.) is selected to boot. The user may be prompted to select what to boot. * Control passes to /System/Library/CoreServices/BootX, the boot loader. BootX loads the kernel and also draws the OS badges, if any. We want to cut in here before it loads the kextscache.mkext and have our hardward detection script modify this file to suit the hardware that has been detected. An other way would be to modify the OSXBundleRequired Info.plist with the hardware detection scrip. I'll be honest I have no idea which would be easiest to edit. but I think the first option would give us a faster boot time than the second. Again I may be wrong. * BootX tries to load a previously cached list of device drivers (created/updated by /usr/sbin/kextcache). Such a cache is of the type mkext and contains the info dictionaries and binary files for multiple kernel extensions. Note that if the mkext cache is corrupt or missing, BootX would look in /System/Library/Extensions for extensions that are needed in the current scenario (as determined by the value of the OSBundleRequired property in the Info.plist file of the extension's bundle. (MoC) how do you propose to get LSPCI to detect the hardware and select the required kexts at start up? I'm not sure how one would go about it myself. Anyway It's 4:44 am I better get to sleep for a few hours I have uni at 9. I'll be posting tomorrow during the day so if everyone can get some ideas together Link to comment Share on other sites More sharing options...
nightwalker12 Posted October 28, 2008 Share Posted October 28, 2008 Amen to that, now, let's get started on this. Rammjet's idea is the best thing ever, because bootable flash drives are easy to manipulate but we're also faced with drivers and compatibility. The way these wide varieties of distributions work on many computers because we're prompted to select our own drivers and applications. We can't install all the drivers/apps because they'll start to conflict with eachother and whatnot. How can we fix this? Well, this reminds me of Linux LiveCDs. They detect your hardware and load the proper drivers for your hardware. Perfect idea isn't it? OS X basically loads all of the kexts it has because there's technically no need for a detection system if you're true to Apple and running a Mac. We're lucky we even have OSx86. So, why don't we build a detection system? Tell the kernel to load our little script, which detects what you have via LSPCI or some other method, and then loads the appropriate kext. Sounds like Virtual Nirvana to me! What do you guys think? ~MoC hi adam,moc,spartango thats a great idea, drivers is the biggest issue and setting up a auto detection system is the best way to go, we have such a wide spread of pc`s to cover for this project where apple is limited to just there own technology, its not going to be easy for alot of the drivers arent even close to being a complete database yet, sound drivers is a big issue with these new chipsets they have out but ive noticed most of the problem is the realtek series but alot of them r getting solved bye some great people here on the forum, the flash drive is good idea for creating this because we can keep editing it until we get it finished, ive worked on linux distributions all the way back to the old days of mandrake, ive made a live install of Beos and linux, done alot of ms work,ive played around with just about all of them i really enjoy osx86 this has shown great progress so far, im using kalyway 10.5.2 and have upgraded it successfuly to 10.5.5 with 99% working sound was my biggest issue still playing with the pins but all in all i only had to use two patches and its working fast and thats what i like fast, ive had alot of reqeusts for a leopard live for graphical apps to be included bye alot of anime and 3d designers they love the osx because it faster doing there work but thats going to be down the road a bit i think, first is the compatibilty of hardware once that is a success we can add what ever we want,ive also had tiger 10.48 installed was working fine but once again the sound issue ive got sound working but it still not 100% when it comes to a live usb flash its going to be a problem when it comes to running this at internet cafes i own one so i no all the head aches of usb witch is the biggest virus transmiter on the market so if were going to use usb live for this reason were going to have to make a safe guard some kind of virus boot protection but even then it wont be 100% full proof for there is new viruses everyday and most of them transmit instantly to a usb ive had a few custumers come with live usb of linux they told me he was at another internet cafe and just wiped his usb clean so we need to consider this area to cd/dvd we dont have problem anyway welcome to the team lets put our heads into a brain strom and make a allsome leopard live Link to comment Share on other sites More sharing options...
spartango Posted October 28, 2008 Share Posted October 28, 2008 hmm...i like the idea of driver detection. As to how to do it...i'm not entirely sure that LSPCI is the way to go. Recall that LSPCI is a user-level program...it definitely has dependencies and such, so we'd need to get a really low-level version cooking to make that work...But definitely not a bad idea at all. I'd like to bring up some of the other parts of the LiveDVD boot and runtime...particularly, loading stuff into ram and interfacing between the readonly dvd and rw rootfs. I'm thinking we should consider something like knoppix/every linux distro does, which is use union fs and have the DVD as a branch, and tmpfs to give everybody the impression that the DVD is rw. That may make the system behave a bit more like the actual disk install or the USB based installations. Dunno.. good ideas all...And lets also start looking into rammjet's ideas regarding boot-132. will add more later...g2g ftm Link to comment Share on other sites More sharing options...
nightwalker12 Posted October 28, 2008 Share Posted October 28, 2008 ya true knoppix might be the way to go it works good with all linux distro, its just a suggestion but it might be a good idea to put a public forum up and get a vote to see what is working for the public the most like boot 123 i no ive read some people have trouble with this but its just an idea Link to comment Share on other sites More sharing options...
nightwalker12 Posted October 28, 2008 Share Posted October 28, 2008 hey guys ive been studying the kernel reading about 19hrs worth so far it looks like it is possible to do a lot of things we want to do with auto detection, it looks like the best way is to make a new kernel and build the library to go with it or just add to the previous, ive just touched the edge on the documentation on this, ill start again tomorrow so if its ok with u guys im going to start building a kernel it might be a good idea if we could set a time to all chat together on irc or of r choice well night guys its 200am here lol got to sleep for a bit Link to comment Share on other sites More sharing options...
(MoC) Posted October 28, 2008 Share Posted October 28, 2008 I'm also working on a small proof of concept detection kernel, it'll be out soon. Was busy with school at the moment. Link to comment Share on other sites More sharing options...
spartango Posted October 28, 2008 Share Posted October 28, 2008 just fyi, OSX does on-demand kernel extension loading(except for basics or non-hw stuff), so we should be able to exploit that when we load up drivers. Also, recall that most/all osx install dvds (i guess osx86) manage to load the drivers they need(albeit generic or not-so-perfect ones) just fine as is. Their model may be a good one. ... ftm, let's get a solid brainstorm session together. about IRC(internet relay chat, see wikipedia)...Let's hook up sometime(time?,day?) on irc.osx86.hu soon. We'll need to consider: Boot process(boot-132/other, and how that'll load the os files or compressed image afterward) Driver Loading(how are we gonna choose the right drivers for everyone/most ppl) Ramdisk (we'll need to research and use modbin's stuff to set that up) Problems from the past(like, stuff that didn't work for ppl like rammjet &why). and more.. Link to comment Share on other sites More sharing options...
REVENGE Posted October 28, 2008 Author Share Posted October 28, 2008 My worry was that regardless of the auto kext detection, we might have conflicts between certain kexts (this stuff comes from everywhere: vendors, devs, usermods, etc). A rather tedious task would be, for example, integrating all the known VIDs and DIDs of stuff that works with different kexts. Link to comment Share on other sites More sharing options...
spartango Posted October 29, 2008 Share Posted October 29, 2008 Right... Conflicts between kexts are generally between graphics kexts and sound kexts.. I'm thinking these can be loaded way later in the boot via kextload(after LSPCI could be used)...I'll try that asap. But beyond those kexts we should be able to get the same sort of drivers working without much work as at least the install disk, which is quite a bit for most machines. Link to comment Share on other sites More sharing options...
skyhighmac Posted October 29, 2008 Share Posted October 29, 2008 Hey guys, I would like to join the team if thats alright with you. About me: I am from AZ (United States). I have a fair amount of experience in desktop apps using applescript. I have some experience with sh and I am always looking to learn. -skyhighmac Link to comment Share on other sites More sharing options...
(MoC) Posted October 29, 2008 Share Posted October 29, 2008 My worry was that regardless of the auto kext detection, we might have conflicts between certain kexts (this stuff comes from everywhere: vendors, devs, usermods, etc). A rather tedious task would be, for example, integrating all the known VIDs and DIDs of stuff that works with different kexts. What if we do an initial auto-detect and then allow the user to input their hardware config at first boot, let the computer write the config to a file, reboot, load, and voila. Then we'd have to dump the Live DVDs and stay flashdrive only. Link to comment Share on other sites More sharing options...
skyhighmac Posted October 29, 2008 Share Posted October 29, 2008 If you had an autodetect, read from an xml file (or something else) with the appropriate kext and device id, you could make that happen, all without writing a config file. Though that would help boot up times if they wanted to run the live dvd more than once w/o waiting for the whole startup. Link to comment Share on other sites More sharing options...
(MoC) Posted October 29, 2008 Share Posted October 29, 2008 Or just do an initial setup? Link to comment Share on other sites More sharing options...
skyhighmac Posted October 29, 2008 Share Posted October 29, 2008 make users run through "is this your hardware, is this correct" (MoC on AIM) That would be a good idea. Then you would have to have to have like Is your video card nVIDIA 7900GT? with a y/n input answer. That could work, but where would we write that config file created to, if, for example you wanted to shut down and start up later? Link to comment Share on other sites More sharing options...
nightwalker12 Posted October 29, 2008 Share Posted October 29, 2008 here guys i have access to all the original documentation and all darwin source code 9.5 if u need anything let me know i can get any original source code u want or the tools KernelProgramming.pdf Link to comment Share on other sites More sharing options...
Recommended Posts