13 replies to this topic

[mrf582]

Do any devs have a software quality checklist that can be used to make a comprehensive checklist for ROM quality assurance purposes? I see so many ROMs here and they all have some issue. It would really help the community if all devs had access to a master checklist for a ROM to be run through before release.  They could then get beta testers to go through and test each item before release. Should lessen the clusterf%$k that is the Razr HD Dev forum.

 

An example would be something like.

 

- Turn on WiFi. Walk into WiFi area. Let it auto connect to WiFi. Walk out of Wifi area and check for reconnection to mobile network.

- Pair Bluetooth headset. Play internet radio. Walk out of bluetooth range.

 

 

etc. Basically various different scenarios to be run through and checked. A Google Doc would let all of us collaborate on scenarios and overtime, it would be a fairly comprehensive list of things to be checked.

 

 

 

Anyone agree/disagree?

[Cooljoe]

Agree

[Memnoch73]

While this is an excellent idea the main issue is not that roms are untested it's that these roms are built for Nexus type phones and ported to ours. They usually have many kernel talking to hardware issues. Some of these are easy fixes where other things may go a long time with very smart devs trying to get things working.

[BytecodeMe]

You guys are ridiculous.

Sent from my Nexus 4 using Tapatalk 4

[cmh714]

You guys are ridiculous.

Sent from my Nexus 4 using Tapatalk 4

And you are very funny!

 

Hope youre doing well Byte.

[RikRong]

Do any devs have a software quality checklist that can be used to make a comprehensive checklist for ROM quality assurance purposes? I see so many ROMs here and they all have some issue. It would really help the community if all devs had access to a master checklist for a ROM to be run through before release.  They could then get beta testers to go through and test each item before release. Should lessen the clusterf%$k that is the Razr HD Dev forum.

 

An example would be something like.

 

- Turn on WiFi. Walk into WiFi area. Let it auto connect to WiFi. Walk out of Wifi area and check for reconnection to mobile network.

- Pair Bluetooth headset. Play internet radio. Walk out of bluetooth range.

 

 

etc. Basically various different scenarios to be run through and checked. A Google Doc would let all of us collaborate on scenarios and overtime, it would be a fairly comprehensive list of things to be checked.

 

 

 

Anyone agree/disagree?

Disagree, wholeheartedly.

 

Almost all the devs do it as a hobby.  They are not paid to develop, they are not paid for their work, and usually, we don't pay for their product.  If we were paying, then I would agree.  However, you choose to flash something voluntarily.  If you choose to flash something, then it's your task to read and research the bugs.  If you don't like something, or something isn't perfect, don't use it.  If you find bugs (not the same as dislikes), then report them.  If you want 100% bug free, go with a stock ROM (heck, those aren't even bug free).  If you want to have the most up-to-date, customizable ROM, then stick with custom and deal with the bugs.  I tell you what, I'm on 4.2.2 based Illusion and I can tell you it's 99% bug free. 

[mrf582]

What bug do you consider to be that 1%?

 

The devs wouldn't have to test for all this. We as a community would, at least those of us that don't mind beta testing. Not all of us want to beta test though... As of now, most of the ROMs have something significantly wrong. Whether it's call volume, or low signal strength, or bluetooth issues etc. And almost none of the ROM threads acknowledge these bugs in the OP.  You have to read through pages and pages of gibberish to find out.

[Memnoch73]

What bug do you consider to be that 1%?

 

The devs wouldn't have to test for all this. We as a community would, at least those of us that don't mind beta testing. Not all of us want to beta test though... As of now, most of the ROMs have something significantly wrong. Whether it's call volume, or low signal strength, or bluetooth issues etc. And almost none of the ROM threads acknowledge these bugs in the OP.  You have to read through pages and pages of gibberish to find out.

 

I stand by my above comments. These are ports based usually on CM10+ so they will have whatever the current CM bugs are. The Devs here that bring these to you are not the original rom Devs or the Original CM devs so that is why ALL that info is not in the OP. As well as CM bugs constantly changing. If you choose to go down the road of those roms it is up to you guys to dig through stuff.

 

We supply the Rom port as well as a place for all those interested to discuss it.

 

Also again these are Kernel issues and will not be easily fixed by just testing...

[jl90]

Disagree, wholeheartedly.
 
Almost all the devs do it as a hobby.  They are not paid to develop, they are not paid for their work, and usually, we don't pay for their product.  If we were paying, then I would agree.  However, you choose to flash something voluntarily.  If you choose to flash something, then it's your task to read and research the bugs.  If you don't like something, or something isn't perfect, don't use it.  If you find bugs (not the same as dislikes), then report them.  If you want 100% bug free, go with a stock ROM (heck, those aren't even bug free).  If you want to have the most up-to-date, customizable ROM, then stick with custom and deal with the bugs.  I tell you what, I'm on 4.2.2 based Illusion and I can tell you it's 99% bug free.

What RikRong says is 100% true. We do this for fun. We do it so other people can enjoy a custom phone experience and not be stuck with what the manufacturers provide. I am a noob to the world of development but I have personally put hours and hours into this. I test the crap out of stuff for myself and others. While I think your suggestion is well intentioned it gives the impression that you do not fully appreciate the fact that you get to enjoy a custom phone experience for FREE. Bugs and all you should be grateful. Thanks.

[bdazzg1]

Do any devs have a software quality checklist that can be used to make a comprehensive checklist for ROM quality assurance purposes? I see so many ROMs here and they all have some issue. It would really help the community if all devs had access to a master checklist for a ROM to be run through before release.  They could then get beta testers to go through and test each item before release. Should lessen the clusterf%$k that is the Razr HD Dev forum.

 

An example would be something like.

 

- Turn on WiFi. Walk into WiFi area. Let it auto connect to WiFi. Walk out of Wifi area and check for reconnection to mobile network.

- Pair Bluetooth headset. Play internet radio. Walk out of bluetooth range.

 

 

etc. Basically various different scenarios to be run through and checked. A Google Doc would let all of us collaborate on scenarios and overtime, it would be a fairly comprehensive list of things to be checked.

 

 

 

Anyone agree/disagree?

If you don't like it, don't flash it! Build your own! 

That's all

[iamhugger]

Do any devs have a software quality checklist that can be used to make a comprehensive checklist for ROM quality assurance purposes? I see so many ROMs here and they all have some issue. It would really help the community if all devs had access to a master checklist for a ROM to be run through before release. They could then get beta testers to go through and test each item before release. Should lessen the clusterf%$k that is the Razr HD Dev forum.

An example would be something like.

- Turn on WiFi. Walk into WiFi area. Let it auto connect to WiFi. Walk out of Wifi area and check for reconnection to mobile network.
- Pair Bluetooth headset. Play internet radio. Walk out of bluetooth range.


etc. Basically various different scenarios to be run through and checked. A Google Doc would let all of us collaborate on scenarios and overtime, it would be a fairly comprehensive list of things to be checked.



Anyone agree/disagree?

. Yes, quoting big post... Usually it is stated within the op what known issues may be.. At time if release.. Not always though!!

XT912 RaZR SpYdEr CDMA

[RikRong]

What bug do you consider to be that 1%?

 

The devs wouldn't have to test for all this. We as a community would, at least those of us that don't mind beta testing. Not all of us want to beta test though... As of now, most of the ROMs have something significantly wrong. Whether it's call volume, or low signal strength, or bluetooth issues etc. And almost none of the ROM threads acknowledge these bugs in the OP.  You have to read through pages and pages of gibberish to find out.

The random freeze that I've had only twice in almost a month of using the ROM.

 

As for beta testing, it gets done.  Most ports going on right now are CM based and trust me, CM beta tests.  As for testing the ports, it gets done.  Personally, I've helped with a couple.  These guys bring you a ROM that makes your phone act like, well, a phone.  All the extras are just that, extras.  If something doesn't work, there's a pretty good chance that it's getting worked on.  Keep in mind, ports are only as good as their base code, and I'm sure devs are getting sick and tired of hearing people beotch about the same "bugs."

[dmoore]

I like it, it does say flash at your own risk, thanks to all the Dev s out there

[Chahta]

I think I'm going to come up with a testing scheme and continually test ROMs and post the results in the respective threads.  Or not...way too time consuming with school and full time work....and a fiance to keep happy.

 

Testing is a nice ideal but not really a reality when your goal is also to keep on the "bleeding edge".  The only ROMs that IMO need to be tested are so called "stable" releases (and RC's leading up to the stables).  If you flash a weekly or daily (or even a monthly) expect bugs.