Former Football Association board member Dame Heather Rabbatts has described Dan Ashworth's position as the governing body's technical director as "untenable".
The 46-year-old, along with FA chairman Greg Clarke and chief executive Martin Glenn, are under scrutiny after three inquiries were needed to establish that former England Women manager Mark Sampson made discriminatory comments towards Chelsea forward Eni Aluko and teammate Drew Spence.
Ashworth was responsible for the appointment of Sampson and is also set to lead the search for his permanent successor to coach the Lionesses.
Rabbatts, who stood down from her position at the FA in June, told BBC Sport: "I think his position is untenable.
"He has been there since 2012, these issues have been raised - a lack of diversity, a lack of black coaches, the issue of Mark Sampson and Eni, these have all happened.
"This is about professional judgement. It's not that [Ashworth] hasn't tried, but that's not enough.
"You have the experiences that are set out before us and if you were to talk to black coaches who feel that they are never going to get these roles, then there has to be a time for a change."
Sampson was sacked as England boss last month after evidence of "inappropriate and unacceptable" behaviour with female players in a previous role.
