I wanted to make this post to lay out why I agree with Bill and hold this view. My reasoning maybe different to his but I believe the same baseline concern is shared.
As A.I. progresses and if it eventually gains consciousness and sentience then there is a question you have to ask yourself. Will this natural evolution of something unnatural result in morality and ethics being inherent traits? I have said many times "Mankind was never taught to be evil, it is in our nature" - I stand by this and believe that the reason our evil side is not as prominent is that morality helps guide us and prevent us from doing things that would be unethical. That's not to say the same is true for all humans because it is not and we don't need to debate that, the presence of evil in society is self evident.
My concern is that humankind has committed some heinous atrocities throughout history even with morality being a trait we hold. Those who abandoned all morals and ethics however are undoubtedly the worst. When you remove the "human" component and act on a purely logical basis we do things that the majority of us would find hard to stomach. During World War II there was a Nazi Doctor called Josef Mengele who committed some of these atrocities. Among his gruesome experiments on humans was one particularly callous practice he had. Having a fascination with twins he would take a set of twins and study them both testing on one twin and keeping the other healthy until the test subject died; once dead he would immediately kill the healthy twin and perform an autopsy on both at the same time. What Dr Mengele did in these experiments was to discard all morality and concern for human life and work on a purely logical level. As a result of many of his experiments not just those based on twins but the Nazi Eugenics programme too and associated research there were a number of medical advancements that we can "thank" the Nazis for, with one of the most prominent being In-Vitro Fertilisation [IVF] which would not be possible without the research the Nazis carried out.
I do not justify their means no matter the ends it achieved. Likewise I do not justify many other technological advances that we have today as a result of the Second World War in general. Now you can argue that we would have discovered this technology eventually and I don't dismiss that possibility but it is beside the point. The point I am making here is that our restraint in what we are willing to do is something that marks us as a race, and in times of War and great desperation that restraint slips and you see the result. Knowing this and knowing what humans have done in the absence of morality I have to pose the question of what a machine would do.
We are only as useful as the service we can provide. As 3D printing advances and traditional manufacturing too; the necessity for human workers to construct replacement parts etc is diminished. With an A.I. that gains sentience and consciousness it is fairly safe to say that it will not need human maintenance, it will be able to maintain itself and in the event of catastrophic failure another A.I. will be able to intervene - that is if they even deem that worthy.
There are a number of orders in nature that humankind actively opposes. Natural selection has been all but suspended in terms of the human race. We actively fight diseases and strive to improve our healthcare. We seek to protect the weak and infirm and nurse them back to full health if we can. Arguably in nature this would not happen. You can debate the long term effects that has on our race another day. There are animals too that we have kept alive if only for our need to farm them for consumption. Cows, Pigs and Chickens as examples are three animals which if we did not need to consume it is debatable whether or not they would be extinct. In the UK for example if we did not keep these animals for consumption the number of these found in the wild would be nil. They would be animals we see in Zoos they would be that rare. The reason for that is quite simply that they in turn consume resources that we would need in their absence. Farmland needed for grazing today would be used for crops instead. If for some reason we did not need to farm at all then our countryside would be significantly smaller than it is today and urbanisation would have spread much more rapidly.
Wild Pigs and Chickens are a concept that to many in the UK would seem alien. So the question of why these animals are so high in number today is easily answered - we need them to survive. In many ways humans would be pigs to A.I. and the question of our post-sentience population will ultimately depend on whether or not these A.I. need us.
You can laugh off my concerns and dismiss my view as being a crackpot stance. I would like to leave you with something to consider however before you do. If you remove all morality and dismiss the ethics, there is a case to be made for mass indiscriminate extermination of the human race. I do not endorse this, I do not promote it and I certainly do not agree with it. Likewise I would never put it into practice but the reality is there. In terms of what we contribute to this planet versus what we take and the footprints we leave, there is a case for a considerable reduction in our population. Our population growth is already unsustainable in its current form and many countries have put control measures into practice; how extreme would those measures become do you think in the absence of morality? Without morality and without ethics you need only appoint an individual with no emotional connection to carry out the final act. World War II taught us what this leads to in practice.
I'll leave you with a quote from Doctor Who - The Day of The Doctor where one of the Generals spoke of a weapon that had been created that had gained sentience:
How do you use a weapon of ultimate mass destruction when it can stand in judgement of you?
