Some of the stuff that they think we're hiding is ridiculous.
Anybody who has read the scriptures, including the D&C, knew that Smith taught on the topic of polygamy and would have also seen the topics of Emma needing to accept it and rules about it - so for those who say in church they were called lairs for suggesting such a thing, I wonder if they were 10 years old when that exchange took place because it's common knowledge.
I think the problem people get is when they accept a wisted version of things. Use polygamy as an example: on one extreme end you still have some Mormons who say that Smith wasn't and some will say that he never had sex outside of his union with Emma; you've got the middle group of people like me who know he had other wives but know that we don't know who he had sex with (nor is it our business) yet also know that he didn't father any children with anybody other than Emma so if he did have physical relationships with others (which would be permitted within polygamy) it doesn't sound like the norm, but then a group of people go way too far over and assume that any polygamist is a sexual deviant in it only for the sex - if you've ever seen real female polygamists I've got news for you, you could go a lot of places and have a lot hotter women without all the hassle if it was only about sex. The problem is that these people, converted to a judgmental point of view, feel that things are being suppressed if you don't get a full deep and dirty version of it - which doesn't logically seem like it was the case anyway. In other words their minds conjure up dirty motivations for others and then those others become the perverts because their minds focused on perverted reasons.
In their mind it's no longer a topic which could be debated with merit on either side, they go from one extreme to the other and claim that those who don't accept their point of view aren't seeing all of the facts - which is rampant cognitive dissonance - which to be fair exists on the Mormon side as well as people accept this notion of absolutes and perfection on the Mormon side, while the anti's become married to a view of absolute corruption and everything is imperfection.
However, these people are few and far between. They pop up in YA on the anti side, but how many are there who frequently post against the LDS church? About a dozen in a world of 7 billion? And they call us the tiny minority! Within the ranks of the church there are a lot of people who won't accept anything that's not perfect in the past with leadership when in reality we don't believe in infallibility of man and there are clearly some blunders were those leaders are looking back at things from heaven going "hey, I thought I did my best, but sure wish I could take that one back", and I'm not pointing at anything in particular only that men make mistakes and men on a platform get pointed at for years for the smallest of mistakes or sins of omission.
The vast majority of Mormons have a somewhat balanced view of things in terms of religion - and the vast majority of former Mormons or inactive Mormons also have a pretty balanced view of things. Most inactive people aren't inactive due to "something hidden" but rather their kids soccer program got in way of attendance and as time went on their attendance went down - that's usually it.
There are those within who do hide, and there are those without who cry that things are hidden, but the majority on both sides don't.