Archive for April, 2009

Weekly update

April 23, 2009

Well, there’s no progress on my gsoc project last week(since last Wednesday ). Roughly summarizing, I spent the first 2 days learning bash shell and some knowledge on administration under Linux. And the last 4 days installing Gentoo, ubuntu and upgrading Lenny to sid. The rest time contribute to my boss and my arm.
The reason I install Gentoo is that I want to get familiar with it since r0bertz has already run optimized Gentoo on Loongson. And I will install the optimized version of Gentoo on it as soon as I got the Gdium Liberty 1000 or Fuloong. I already registered in the OLPH group and ordered a new Gdium Liberty 1000 but no reply yet.
I had a small surgery on my left upper arm last Friday. It’s a “open drainage” operation which means cutting a bump and getting pus out of it. The time is just 3 minutes however, I feel like 3 years!! The anesthetic didn’t work at all I think because the damn doctor cut my arm as soon as he finish injecting the anesthetic, not waiting for it to take effect. So I spent the following three days whimperring and having intravenous injection.
Since I don’t want to try new distribution now and my arm become better, I plan to do these things next week :
1. Try qemu with Debian mipsel port and CLFS with qemu.
2. Keep learning mips ISA
3. Learning tools such autoconf, automake etc
4. How to maintain a port in Debian[1]

[1]http://www.debian-ports.org/

Advertisements

Distribution Syndrome

April 23, 2009

I’m not sure if I have this “Distribution Syndrome” these days.
But I installed gentoo, ubuntu in my lap-top in the past 4 days, both of which are my first try. Gentoo cost me 2 days to have a gnome desktop and ubuntu 9.04, both of gnome and kde installed, cost me half day today. Of course I love Debian best, but I have to say that both of them are very good.
Gentoo is great in installing and customizing packages. The info displayed during installation is very intuitive that if a failure happened, you just check the build log it generated, and most of the time you can solve it by manually installing packages with suggestions it provides, such as adding additional USE flag. Unlike Debian which use many package management softwares like dselect, apt-get, aptitude and dpkg , all having their advantages and disadvantages, in Gentoo there is only one and a great
one–ebuild. So you just need to learn as hard as you can about it, then you treat packages whatever you like.
As for ubuntu, it’s better than winXP IMO. Gnome + compiz are the coolest desktop appearance I’ve ever seen. And KDE is really beautiful and smart. Lots of widgets can be put on your desktop to make your life colorful.

I have to stop this syndrome since gsoc bonding period has already begun. I can’t waste my time to keep trying new distro any more though there is a lot fun:-) So opensuse, arch and foresight, I will not touch you in the near future but I’m watching you!

Weekly progress

April 15, 2009

A week passed since the last post. However, I feel it’s such a long week that I can’t even remember the feelings or something when I wrote the last post. Possibly because I’m too tired these days since I’ve not slept well for a month. I doubt whether I could live like anymore.

Good news is my GSoC project is one of the 11 Debian’s GSoC projects that MAY have been get funded. I say MAY because the official results will come out at April, 20th and as said by Arthur, “anything can change”. I hope that’s just one of Arthur’s joke.

Last week my boss assigned me a task that I’ll write a blif parser by this week and on Tuesday there is a seminar at which it’s my turn to give a presentation. As usual, the subject of the seminar could be based on papers published recently on top micro-electronics conferences or journals, or something interesting which will “benefit us directly” or “broaden our view’s about the world”. I thought it’s a great idea to present something about “free software and open-source movement”, however, the darn parser I mentioned above cost me lots of time and it’s Monday morning when I realize that I haven’t prepared a single slide! So I spent the afternoon searching free software stuff, hoping there’re some available PPTs or lectures fit my scope. However, I found nothing and embarrassingly,  I found that I can’t understand some philosophies stated on gnu’s website. For example, the one Why Software Should Not Have Owners, which I read for half an hour and still confused. So I can’t give a presentation on sth that I’m not convinsed and I had to resort to my “secret weapon”–SEU(single event upset) which I had some research months ago and I ended in giving a boring presentation next day. I badly hope that I can UNDERSTAND these “selected essays” wrote by Rechard Stallman the next time I’m on duty.

Lots of crap above. Now summarize things done this week:

1. Successfully build a minimal LFS based on the very detailed lfs-book. I gained some insignt into linux though many procedures are not fully appreciated, especially the part with “bootstrapping”(gcc and glibc), sometimes using these libs sometimes with others etc. And one of the very important lessons I learned is never ever adjust time when compiling, especially adjusting time back. This will cause the make command  to stuck in a deadlock since some files’ time stamp is in the future. I learned that when i realize my new linux system is one day ahead of time and adjust it back when I compile the Glibc to build the basic system. And long time passed when I realized the make were doing things repeatedly.

2. Studying mips by watching see mips run linux. Thanks to Rayson, I download this book on internet. The mips ISA is quit different from IA-32 , which I understood easily after learning the book Computer Systems: A Programmer’s Perspective, that it’s a little hard for me to read this book. Maybe I should read some chapters from the books witten by Hennessy and Patterson first(like Computer Architecture : A Quantitative Aproach and Computer Organization and Design ).

3.Some tiny things like, learning vim, shell script etc. vim is much better than I thought since I always use emacs. I think vim is more convenient and smart than emacs in editing shell script or config files(administrator side) while  emacs is more powerful in developing large software(programmer side). I like both of them( I hope i’ll not confused the key bindings)

Things to do next week:

1.Populate the minimal lfs system with X , network capacities and other applications. This job can be guided by the BLFS manual.

2.Start building a CLFS(need to clear more space in my laptop hard drive first)

3.Reading mips book. Understand the register use conventions, mips assemble language

4.Talk to foka or r0bertz and ask their opinion on what to do next.

Well, r0bertz start submitting patches to kernel with regards to Loongson 2F these days and get Ftrace working on his computer. I’d better hurry up in case the gap between him and me is so large that I can’t reach him anymore.

Waiting For Results

April 8, 2009

It seems foka were really busy recently. I met him on #debian-soc yesterday and he told me I’ve got another ‘sponsor’–Lazyfai. I was very excited since foka mentioned him once that he has already do some work on building a N32 ABI MIPS port for Debian. No time for enjoying this great news with me, foka just disappeared, leaving me alone grinning like a Cheshire cat…

However, right now is a perfect time for me to warm up. Although the proposal result or even the shortlist doesn’t come out yet, it’ll be a waste of time to just wait for these things doing nothing. And furthermore, I’ve already made up my mind that whatever I got passed or not I’ll devote myself to this project in the summer. It’s my first show in FOSS world and I have the chance to learn things and being mentored by people like foka and r0bertz. HOW CAN somebody just give it up!?

So I think I’ll do these things down below as much as possible before I got further instructions from mentors:

1.Read <See MIPS Run> (already borrowed one today)

2.Follow the guide to build a lfs and clfs system(lfs is 50% done today)

3.Do research on r0bertiz’s pioneering work[1]

I think these 3 things are enough for me to get no sleep these days. However, there are so many thing to to like:

1. Talk on debian-mips. Ask their opinion on this project, especially on building a new branch from MIPS port. ( already sent email, no reply yet)

2.Get to know about loongson 2F ISA and features. Gathering info from official data sheets and maybe more importantly  from the changelog of gcc-4.4, binutils2.19 and glibc2.9. Information about PLT(procedure linkage table), PIC(position independent code) and “h asm constraint”(??) will be highlighted.

3.Preparing for benchmarks and N32 ABI apps. Since the results from [2] show that a least one of the huge improvement from “march=mips1” to “march=mips3” is from audio CODEC test(Loongson = 230~240 seconds, mips1 = 500 seconds).Also, in their test, N32 ABI test result is missing duo to incompatibles. So provide enough N32 application is also a job.

4.Get to know buildd system if I have to host a mirror of Debian and this job come along with many more things to care about, things I can’t describe or predict right now(But I am SURE that there are so many such things!) such as how to host a mirror, how to access a remote host etc.

Well, for now, just these things above are within my short-sighted eyes and I’m not really sure what I saw is what it is. And time goes by, more and more new and stronger monsters will show their bloody teeth. So this job is really a challenge for me, a challenge lasting for 4 month to deal with and a challenge I believe will benefit me a lot if i succeed. *Very* luckily, I got r0bertz, foka and Lazyfai behind me!

[1]http://www.lemote.com/bbs/viewthread.php?tid=20125&extra=&page=1

[2]http://www.loongson.cn/company/techonology_info.php?id=25

GSoC Proposal: Creating a new MIPS N32 ABI port for Debian

April 5, 2009

Creating a new MIPS N32 ABI port for Debian

Abstract:

This project first focus on creating a new MIPS N32 ABI port for Debian. Different from O32 and N64, N32 is a address model which has most 64-bit capability but using 32-bit data structure to save space and process time a lot. A second focus will be given on making such a “mipsn32el” arch fully optimized for Loongson 2F CPU[6] which gains more and more popularity in subnotebook/netbook.

Background:

I’m now pursuing my master degree in Fudan University in Shanghai. I major in Micro Electronics and choose EDA software development as my research interest. I have experience with C/C++ programming in a FPGA EDA software project in my lab, in which I’m in charge of : a verilog parser using flex/bison; a netlist viewer using MFC. I’ve taken part in a Altera held competition with 2 other students in which we design and implement a “wireless multimedia monitoring system” which won us an outstanding reward finally. I have also some experience in porting linux to Xilinx and Altera’s FPGA development board although I just mainly copy other people’s pioneering work.

Besides the technical background, the best reason I give myself as the best person for this project are is I’m extremely fascinated by the open-source software development and joining in GSoC project is a precious opportunity to learn about it, including from mentors, developers and other students like me. Another reason worth to be mentioned is, I’m very proud to port one of the most popular and also my favorite linux distribution to Loongson, which is all designed by my own country. Loognson will not only benefit from the large amount of packages in Debian, but also from its talented developers and its influence in FOSS world.

Synopsis:

In this project, there are 2 main tasks. First, create a MIPS N32 ABI port for Debian. As mentioned above, N32 ABI model is the best for current 64-bit MIPS arch. This port will use “-march=mips3 -mabi=n32” to make it more generic. The second goal of this project is to optimize the generic port for Loongson 2F with “-march=loongson”. Make such optimizations needs new feature supported in the latest tool chain, to be exactly, gcc-4.4, glibc2.9 and binutils2.19. It is reported that this optimization will fully utilize Loongson 2F’s 64-bit capability and push its performance to utmost [1]. Of course benchmarks will be provided to give a thorough comparison between O32, N32, N64, not only on loongson’s behalf but also will provide datas as reference for other MIPS systems.

Benefits to Debian:

With the growing popularity of Loongson 2F based computers such as Gdium Liberty 1000 Netbook[3], Lemote Yeeloong 8089 Notebook[4] and Fuloong[5], there is a great need and interest to have Debian running at top performance on these machines, especially to serve the greater cause of promoting free/open-source software and of providing computer-assisted education to those who most need it. This work will also make Debian more “universal” by adding a new arch and more popular through its use in Loongson based computer that it’ll attract more talented people to contribute to Debian.

Deliverables:

The first three should be achieved with no excuse while the last two tasks especially the last one are optional. I list them here in case I can assure the quality of the first 3 tasks and there are time left for them.

  1. A minimal basic system of Debian fully optimized with N32 ABI
  2. Make this basic system optimized for Loongson 2F
  3. Providing benchmarks to compare performance among O32, N32 and N64 port. The comparison will not only take care of compilation time but also actual system performance as a whole.
  4. Migrating X windows and basic desktop environment
  5. Migrating application such as Iceweasel and OpenOffice.org

Project details:

As discussed with in IRC with foka, marga, r0bertz, aurel32 etc,  it is better to build a “generic” N32 ABI port at first and then we can use benchmark to fully study its performance compared with O32, N64 and N32 ABI optimized for Loongson 2F. I list jobs to be done down below:

I will start the porting just like others by following CLFS instructions to build a minimal system with N32 ABI by using “-march=mips3 -mabi=n32”. The “mipsel” port in current Debian repository is multilib version but mostly optimized for O32 which is very old. This “generic” port will push performance of those 64-bit MIPS systems to limit.

As such a minimal system is established, optimization for the Loongson will begin. The optimizing job is mostly done by a series of tools including gcc4.4, glibc2.9, binutils2.19. These version is selected because some features are supported in them. For instance, gcc4.4 is optimized for loongson by supporting its instruction set, pipe-line model and scheduler support etc. and  glibc,binutils support PLT etc.  I, on the other hand, will  focus on finding incompatibles.

Then benchmarks will be provided to give a thorough comparison between O32, N32, N64, not only on loongson’s behalf but also will provide datas as reference for other MIPS systems. There is a convincing report by r0bertz that said “coreutils compilation time reduced from 19 minutes to 12 minutes” and “30% overall speed improvement”. So the benchmarks should test against this report and besides these results, more tests should be run to treat the system performance as a whole instead of just choosing compilation time.

If everything goes smooth,the rest of the time will be devoted to porting X, basic desktop environment and other applications. Porting applications should be a collective job in community rather than a single person’s work. Making this job explicitly public will attract more people and will accelerate the pace of porting Debian to Loongson remarkably.

As above mentioned, r0bertz who likes to co-mentor this project with foka, has already pioneered this work using “-march=loongson2f -mabi=n32 -mplt”[1][2]. His work can be a good starting point for this project since he has already submitted many patches and applications for users to test. I think his opinions will help me a lot in this project.

Project schedule:

These timeline are fairly simple and inaccurate, however I’ll be very glad to change it with advice from mentors and other people caring this project.

  • Preparation Stage, April 20th~May 10th : Get myself started and finish Deliverables(1).
  • Mid-term stage, May 11st~July 5th : Deliverables(2)
  • pre-Final stage, July 6th~Aug 13th : A). Deliveralbes(3). The first 3 systems to compare include : the generic N32 version, the version optimized for Loongson and the “mipsel” multilib version Debian already have.B). Release a ‘testing’ version of Loongson based Debian on the internet to let volunteers in the community to try and test. C) Deliverables(4) and (5) if time allowed.
  • Final stage, Aug14th~Aug 23rd : Fix bugs, send patches, document the project and submit it to Google.

Other summer plans:

None, really. Idling on internet or playing wow if it’s counted.

Exams and other commitments:

No exams anymore. A little work involved with TA though.

If you are a Debian Developer:

Not yet.

If you are not Debian Developer:

Actually I’m the only Debianer in my whole lab(>100 students, lots of sub-labs), amazing huh?! Considering RH is not free, less packages and Ubuntu as a unstable version of Debian, there is no obvious reasons to ignore Debian. As a firm fans of Debian, I feel the need to promote the popularity of it, first in my lab and then on internet. As a relatively young Linuxer, I may not help people to solve their complicated problems but I can do tutorial jobs(like posting HOW-TOs) easily and happily. IMO, getting Debian more popular in China will be a very attractive idea since the user base in China is very large and due to this fact, more potential contributors will be led to FOSS world due to the effort Debian has made.

I was very lucky to find this project almost near the end of application. This project is a very precious chance for me in two reasons. It will provide me a environment in which I can learn a lot of interesting things and cool skills from mentors, developers and other students. I treat this project not as a assignment to finish but as a door leading to the great un-known world. Besides, the project involves Loongson 2F cpu which every Chinese people is proud of. And if i could port Debian, which is the best Linux distribution in my opinion, to Loongson, this summer will definitely be the most unforgettable summer in my life. So I’ll be very very happy contribute to this project in and after this summer.

Reference:

[1]: http://www.lemote.com/bbs/viewthread.php?tid=20125&extra=&page=1

[2]: http://dev.gentoo.org/~redhatter/mips/loongson/

[3]:http://www.geeky-gadgets.com/?p=9968

[4]:http://www.tekmote.nl/epages/61504599.sf/en_GB/?ObjectPath=/Shops/61504599/Products/CFL-001

[5]:http://www.lemote.com/english/fuloong.html

[6]:http://www.lemote.com/english/cpu.html

My Proposal

April 4, 2009

Well, I finally submitted my proposal last night in the LAST MINUTE!
6 hours before deadline, I submitted my first draft proposal. 2 hours before deadline, I have a really enlightening discussion with foka, aurel32, r0bertz, marga etc. They suggest me to change the name and most importantly they suggest I should create a generic N32 ABI porting instead of just optimized for Loongson 2F. And then we can make hardware specific optimization for Loongson 2F later using benchmarks to see how much improvement can we get compared with N64, O32 ABI. I think that is really a good plan so I decide to modify my proposal. And after taking foka’s kind-hearted advice on some spelling checks and grammar, I noticed that deadline is just 45 min away….

Well, the rest can be figured out by most people. I’m just like “热锅上的蚂蚁“(An ant on a very hot pot). I rush to modify the title and abstract since they’re the part seen by most people. And I save my proposal every time I change it a little in case google is impatient to wait the last few minute….However, at last I still failed to mention Debian-mips mailing list foka told me and leave my proposal kind of a hybrid someway…

Anyway, I’ve got a few days to comment under my proposal to clarify things and no rush this time I hope.

Down below is my proposal

Abstract:
This project first focus on creating a new MIPS N32 ABI port for Debian. Different from O32 and N64, N32 is a address model which has most 64-bit capability but using 32-bit data structure to save space and process time a lot. A second focus will be given on making such a “mipsn32el” arch fully optimized for Loongson 2F CPU which gains more and more popularity in subnotebook/netbook.
Content:
Proposal

Name: Sha Liu

Email: sandyleo26@gmail.com;

Background:
I’m now pursuing my master degree in Fudan University in Shanghai. I major in Micro Electronics and choose EDA software development as my research interest. I have experience with C/C++ programming in a FPGA EDA software project in my lab, in which I’m in charge of : a verilog parser using flex/bison; a netlist viewer using MFC. I’ve taken part in a Altera held competition with 2 other students in which we design and implement a “wireless multimedia monitoring system” which won us an outstanding reward finally. I have also some experience in porting linux to Xilinx and Altera’s FPGA development board although I just mainly copy other people’s pioneering work.

Besides the technical background, the best reason I give myself as the best person for this project are is I’m extremely fascinated by the open-source software development and joining in GSoC project is a precious opportunity to learn about it, including from mentors, developers and other students like me. Another reason worth to be mentioned is, I’m very proud to port one of the most popular and also my favorite linux distribution to Loongson, which is all designed by my own country. Loognson will not only benefit from the large amount of packages in Debian, but also from its talented developers and its influence in FOSS world.

Project title:
Creating a new MIPS N32 ABI port for Debian

Synopsis:
This project first focus on creating a new MIPS N32 ABI port for Debian. Different from O32 and N64, N32 is an address model which has most 64-bit capability but using 32-bit data structure to save space and process time a lot. This is a generic MIPS-III N32 port so modify it in different endian system and make optimization in some specific N32 MIPS arch is only a trivial thing. The second goal of this project is to make such optimizations in Loongson 2F CPU which is a MIPS architecture cpu developed at Chinese Academy of Science. It is reported that this optimization will fully utilize Loongson 2F’s 64-bit capability and push its performance to utmost [1]. The work will start from porting a minimal version of system and then extended to porting X window system, basic desktop environment and finally start porting useful applications. There is a long way to go to finally build a full mirror of all Debian packages, however my work will only focus on the most initial and basic part of this whole project.

Benefits to Debian:
With the growing popularity of Loongson 2F based computers such as Gdium Liberty 1000 Netbook, Lemote Yeeloong 8089 Notebook and Fuloong, there is a great need and interest to have Debian running at top performance on these machines, especially to serve the greater cause of promoting free/open-source software and of providing computer-assisted education to those who most need it. This work will also make Debian more “universal” by adding a new arch and more popular through its use in Loongson based computer that it’ll attract more talented people to contribute to Debian.

Deliverables:
The first three should be achieved with no excuse while the last task seems to unlimited. So I just list two well-know apps and will work more on the rest if time allowed:
1).A minimal basic system of Debian fully optimized with N32 ABI
2).Providing Benchmarks to measure improvement between fully optimized version of Debian and mipsel version, which also evaluate system performance as a whole instead of just compilation time[1].
3).X window system and basic desktop environment
4).Migrating Iceweasel and OpenOffice.org

Project details:
As discussed with in IRC with foka, marga, r0bertz, aurel32, it is better to build a generic N32 ABI port at first and then we can use benchmark to fully study its performance compared with O32, N64 and N32 ABI optimized for Loongson 2F. I list jobs to be done down below:

I will start the porting just like others by following CLFS instructions to build a minimal system with N32 ABI optimized. So that will include : building cross-compiled toolchain(gcc, glibc, binutils etc.) and a temporary minimal linux system(more utilities such as Bash, Ncurses etc. ) by using the toolchain generated above. Besides that, kernel, bootloader and other few utilities should also be built since I’ll work under x86 and build target system for mips architecture. Another thing to note is that since gcc-4.4 is optimized with Loongson CPU such as providing instruction, pipe-line model and scheduler support, using this version in the target system is an obvious choice though it’s not released officially yet.

After a minimal system is installed, benchmarks should be provided to show the improvement caused by the optimization. There is a convincing report by a Gentoo developer and also a Loongson guru, said “coreutils compilation time reduced from 19 minutes to 12 minutes” and “30% overall speed improvement”. So the benchmarks should test against this report and besides these results, more tests should be run to treat the system performance as a whole instead of just choosing compilation time.

The rest of the time will be devoted to porting X, basic desktop environment and other applications. Porting applications should be a collective job in community rather than a single person’s work. Making this job explicitly public will attract more people and will accelerate the pace of porting Debian to Loongson remarkably.

As above mentioned, a Gentoo developer has already pioneer this work using “-march=loongson2f -mabi=n32 -mplt”[1][2]. This can be a good starting point for my work since he has already submitted a few patches and applications for users to test.

Project schedule:

These timeline are fairly simple and inaccurate, however I’ll be very glad to change it with advice from foka and other people as time goes by.
Preparation Stage, April 20th~May 10th : Get started and porting a minimal version of Debian to Loongson optimized with N32 ABI.
Mid-term stage, May 11st~July 5th : 1).Compare the systems by providing benchmarks to run under different arch(mispel) or address model(O32 or N64). 2).Migrating X and Basic desktop environment and other utilities to the system.
pre-Final stage, July 6th~Aug 13th : 1).Release a ‘testing’ version of Loongson based Debian on the internet to let volunteers in the community to try and test. 2).Track bugs and fix them. 3)Port other useful packages.
Final stage, Aug14th~Aug 23rd : Fix bugs, send patches, document the project and submit it to Google.

Other summer plans:
None, really. Idling on internet or playing wow if it’s counted.
Exams and other commitments:
No exams anymore. A little work involved with TA though.
If you are a Debian Developer:
Not yet.
If you are not Debian Developer:
Actually I’m the only Debianer in my whole lab(>100 students, lots of sub-labs), amazing huh?! Considering RH is not free, less packages and Ubuntu as a unstable version of Debian, there is no obvious reasons to ignore Debian. As a firm fans of Debian, I feel the need to promote the popularity of it, first in my lab and then on internet. As a relatively young Linuxer, I may not help people to solve their complicated problems but I can do tutorial jobs(like posting HOW-TOs) easily and happily. IMO, getting Debian more popular in China will be a very attractive idea since the user base in China is very large and due to this fact, more potential contributors will be led to FOSS world due to the effort Debian has made.

I was very lucky to find this project almost near the end of application. This project is a very precious chance for me in two reasons. It will provide me a environment in which I can learn a lot of interesting things and cool skills from mentors, developers and other students. I treat this project not as a assignment to finish but as a door leading to the great un-known world. Besides, the project involves Loongson 2F cpu which every Chinese people is proud of. And if i could port Debian, which is the best Linux distribution in my opinion, to Loongson, this summer will definitely be the most unforgettable summer in my life. So I’ll be very very happy contribute to this project in and after this summer.

Reference:
[1]: http://www.lemote.com/bbs/viewthread.php?tid=20125&extra=&page=1
[2]: http://dev.gentoo.org/~redhatter/mips/loongson/

My First Blog Abroad

April 2, 2009

really cool! The style, the layout, the fonts etc….Why my hometown blog look so dull?

Not really much to input today, just come here to leave a footprint. Hope my  draft  “of ” (draft)+  “of” proposal will be on time tomorrw and got a good result I hope. Anyway, I should feel really lucky and grateful about what i’ve got by now. I think I’ll publish my form thanks after the results comes out.

 

But all before that ,thanks Arthur for introducing me here!

Hello world!

April 2, 2009

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!