I am a Sign Language student who has been learning Virtual Reality ASL for ~6 months. Ask me anything!

Someone downvoted all my answers to the questions for some reason lmao

Edit: Now some questions have been downvoted too. Great.

Reddit is a very strange place. I chalk it up to bots (although I don't understand the motivation) and very small-spirited people who get their jollies by trying to pull down everything around them. Pay them no mind.

I won't, thank you c:

Well, it’s also helps to have a healthy (I said, healthy) amount of skepticism.

It’s no secret that high karma accounts are for sale, and once that little baby icon (Apollo shows new accounts with a baby icon to help look for karma farming) goes away, a name like ThrowAwayMcgee is a great name and could be seen as “tongue-in-cheek” when that account is sold.

Sure, he could be genuinely not wanting to mix accounts, and could genuinely be a nice person trying to learn; but the odds are against OP. The amount of little baby icons posting high karma farming posts is just too damn high.

What are you insinuating I'm doing here?

What kind of controllers are you using that give you the ability to sign? I remember only being able to make fists and peace signs

I'm using the Quest 2 controllers. With a basic avatar you have the ability to have: An open hand, a relaxed hand, a fist, index finger, thumb, L hand, Peace sign and rock n'roll and the language has been altered to fit with these limits. There are also certain avatars that can give you more handshapes using combos. Mine gives me F hands, R hands, X hands, Middle finger down and flat O's but I've seen others with more, like the bent Peace sign used for "Pizza", "Blind" and "Potato" or the K/P handshape. Anyone with any kind of vr controllers can sign.

Would the hand tracking work or is it too janky?

Not only does the hand tracking not work for VRchat (unless you're using virtual desktop) it's way too bad regardless. If you have a quest try to track you putting your thumb between your index and middle fingers, that's the letter T in ASL. Have fun watching your quest freak out, lol. Now imagine it trying to figure that out when your hand is facing whoever you're talking to.

So what's the point? Why not just type in game?

Because it's rare that a world has a place to type, signing is faster, it's more enjoyable and it's started a community of 3.5k people. The thing about the game is that every single place you can go the person who made said place would have to add a keyboard on the wall or something which just doesn't happen.

Fair enough. I would have thought they could add a little wrist option and use touch typing in vr like the other games I've played but whatever is easiest I guess

VRchat only adds things that won't be useful, sorry mate.

do you think altering the langauge to fit within the parameters of the VR equipment is detrimental to your overall understanding of the langauge?

"The language" being real ASL or VR ASL?

real ASL

I mean I've never really used true ASL so I wouldn't know

Wait, so are you learning ASL specifically to talk in VR? I'm so confused why one would do this when microphones exist. Or are you deaf and just never learned ASL?

I'm hearing but I have Deaf friends in vr. I'm also Semi-Verbal due to autism so I stop talking if I'm stressed. Also it's just nice to sign with other hearing people.

Oh okay that makes sense then My bad when I commented your OP had been removed so I didn't see the context.

Fair enough

So you are still vastly limited.

We have limits but we've worked around them and have either changed signs to be possible or changed our hands to do the signs.

Have you been able to converse with the deaf to help hone your skills? (I was an interpreter years ago and I always found it took me a little time to become accustomed to a person's signing - almost like accents or dialects for the hearing - especially after learning on one coast then moving to the other.)

Yeah, there are tons of Deaf people in the community. I can definitely relate to the sign accents being difficult, but you tend to get used to it and I'm certain it's not as varied compared to real life. I think most of the teachers are from the East Coast so it's probably closer to Eastern signing than anything.

Edit: By "Tend to get used to it" I mean you get used to everyone signing slightly differently from each other in VR

What's the happiest memory you have of your childhood? Something that you'll never forget and never fails to make you smile.

I don't really remember my childhood much and nothing comes to mind

Why would you need VR sign language if you could just put words into chat?

Most places don't have a chat and when it does sign is much faster.

Which sign language?

ASL - American Sign Language. I do know the alphabet I'm BSL/AUSLAN though just the alphabet isn't very good. I was planning on learning more but decided on sticking with ASL for now.

There are ASL, BSL, DGS, LSF and KSL classes in the Helping Hands community and they're all free to attend.

BSL = British Sign Language

AUSLAN = Australian Sign Language

DGS = Deutsche Gebärdensprache (German Sign Language)

LSF = Langue des Signes Française (French Sign Language)

KSL = Korean Sign Language

Yeah, I practice the BSL alphabet now and then, though I still have to do them in order to get them all. IE if you ask me out of no where to do the sign for G or something, I would need to do a run up from A. 😀

I studied ASL for 4 years in college, but certainly never did in VR.

Are you British or American? Studying ASL makes me think American but practising BSL makes me think you're a Brit.

Real ASL isn't as easy as VR, for me it's mostly the facial expressions that I can't wrap my face head around, but it's more complicated because you have full control of your fingers IRL.

Canadian.

One of my friends is a former Auslan interpreter who moved to Canada. She was learning ASL with me, but knowing her gave me an interest in at least learning the alphabet. It's an interesting difference, but I do like ASL better (except 'R'...too close to 'U')

Oh, that makes sense. I was confused why an American would want to learn BSL but I was concerned that someone in Britain would study ASL for 4 years instead of BSL lol.

In VR we don't have the R/U problem because we can't make the U handshape regularly, so to sign U we just move the V hand in a U-shaped-swoop-type thing.

Actually, the only letters that are the same in vr are C, D, L, O, V, Y and Z, but special avatars can give us access to some others. Usually nothing more than R, F, X and I, though.

Can I join if I don’t have the VR?

You can! You won't be able to sign obviously (You won't have hands) but you can download VRChat on a PC and visit ASL worlds and talk to people there. The majority will be able to hear you but if you want to talk to a Deaf player you'll need to ask someone to interpret for you.

✋👌👉👍🤜🤘. 🤘✋👋🖕👇👏👇👇 👉🤘👆?

I know you're joking but I'll do my best to translate lol

"YOUR F G THUMBS UP O [indecipherable] YOUR HELLO F*CK YOU Q MARRY Q Q G [indecipherable] L?"

🤟 ❤

Lmao

'Tis a tale that will transcend all known story-telling and be told for generations

Socks with sandals, yes or no?

Generally wearing anything on the hands makes signing difficult but 2 items made specifically for the feet will probably make it impossible. After a long time thinking about it I'll have to go with yes. Seems like a fun challenge.

How fast can you sign? (Like would you be able to sub someone speaking in real time?)

Even professional interpreters can get left behind a little so I definitely wouldn't wouldn't able to translate speech into sign fast enough. (Though I do do this often, just not keeping up with the speech)

I have more luck with translating ASL into speech though, as I have tons of practice translating for friends who either can't or won't speak verbally.

Also, I can Sim-Com pretty naturally.

(Sim-Com means simultaneous communication and it's just when you sign and speak at the same time so the Deaf and hearing can both understand)

Just an FYI, translating is for written word. Interpreting is used when it’s spoken word.

I tend to use translate when referring to myself because I don't want to even give anyone the faintest idea that I'm an actual interpreter

Just because you’re not doing it professionally doesn’t mean you can’t use the correct terminology! I thought you contextualized it well with your use of “professional”. Anyone who interprets spoken word, regardless if they are an amateur or a professional is still interpreting in the literal sense of the word.

Fair enough

Wait, is your name the acronym CODA?

Indeed it is. Both my parents are Deaf, and I’m a fluent native ASL user. I also interpreted professionally for about 5 years or so.

I didn't even notice your name, lol. What's your opinion on VR sign, seeing as you've had so much experience with true ASL?

Well aside from this thread I actually never even knew about it. It’s definitely no replacement for the real thing (yet) due to the limitations of the controller and not being able to use your hands. Like the physical movements you’re doing with your hands on the controller I imagine don’t directly translate if you were to do the same thing without the controller kind of thing. So the VR manifestation may be equivalent to IRL signs but you’re not really quite developing the same motor skills (but are definitely improving your receptive skills, which I believe you mentioned in another comment it’s easier for you to perceive than to produce).

Also facial expressions are such a huge part to the grammar but I found it interesting how you mentioned users are getting around this limitation: head tilts to the side to show questioning vs the traditional raising of eyebrows kind of thing. I love it from an exposure/awareness perspective too. Always good to build hearing allies and get people interested in ASL or deaf culture. It’s a win if this leads someone (maybe you? Lol) to becoming an interpreter for example.

I do sign IRL to myself as well which does circumvent the whole muscle memory thing, as well as letting me think clearer because I focus only on what I'm signing at the time rather than getting distracted.

The movements you make with your hand are roughly the same as what you'd do without a controller but definitely not similar enough for it to be good practice.

I've met many people in VR who have decided to become ASL interpreters, which is really cool. Now it's up to them to make it happen.

I have genuinely considered becoming an AUSLAN interpreter and I haven't stumbled upon a job title I like the sound of more than that, so... maybe c;

Serious question, how is sign language better than text? I imagine it only being used IRL, otherwise texting should be just easier?

Copy & Pasted from a different answer

Because it's rare that a world has a place to type, signing is faster, it's more enjoyable and it's started a community of 3.5k people. The thing about the game is that every single place you can go the person who made said place would have to add a keyboard on the wall or something which just doesn't happen.

When I took ASL 1 in college, they said a good chunk of the communication is in the facial expression. Do you run into any issues because of the lack of seeing the face, and what do you do to compensate?

Yeah, lack of facial expression can be challenging but you can use body language a lot. Like tilting your head when asking a question or looking around in astonishment when confused. We're also very energetic while signing, like if we need to say something is REALLY GOOD we can sign "VERY VERY VERY GOOD" really energetically.

If you could, would you send raisins to the moon?

I'd rather send an Australian flag

Just to mess with people

Is it OK to finger spell a lot? I taught myself finger spelling but can't get the deeper language.

Yeah, that's fine! IRL Deaf people will just he super happy that you made an effort and fingerspelling can let you communicate whatever you want, albeit slowly.

In VR it's not uncommon for those new to the language to fingerspell almost everything and only know 15-50 signs for a while but after enough time it just becomes as easy as talking unless you're trying to explain something complicated. (I had a lot of trouble trying to convey my confusion and dismay at how seriously kangaroos damage cars when you hit them.)

Do you have a deaf name, can you explain how you got it and how one gets one?

So sign names are like regular names... but signs. They can just be the initials of someone's name but sometimes they get really creative. Someone I know had a teacher who would always stroke his beard in a particular way, so copying that was his sign name. Sign names are given by Deaf/Hard of hearing people and cannot be given by hearing people. Getting a name sign can either be a regular occurrence or a badge of honour depending on what kind of community you're involved in, and it's always considered rude to ask for a name sign.

I do have a name sign! The avatar I use has rebbit ears and my username starts with an L so my sign name is "Bunny" but done with the L handshape. I like it - it's adorable.

Do you think VR is a massive difference to just using videoconferencing when it comes to using and teaching sign language? I often hear that VR meetings would be boring, but I think that's just misplacing what VR meetup could do. I feel that the use of sign language would be the best example, as it would do something that is better in 3D than in a small Skype webcam image.

If the video quality was bad you wouldn't be able to understand the sign which is the biggest change I think.

The second biggest would be that you can see the signs in a 3D environment rather than a 2D one and that can help comprehension of the signs more.

An ASL tutor program was a class project of mine when I did my Study Abroad in the states.

I was using the leap motion controller which records your hand positions.

https://youtu.be/_d6KuiuteIA

https://youtu.be/QrGg6eOiy4E

How is the accuracy with normal VR controls, or do you use other interfaces such as a kinect, other camera, or motion controller?

Unfortunately while we could record correct signs we never figured out the recognition - regardless of what we tried it never worked out correctly.

I'd intended to continue it when I got home but unfortunately my laptop got literally destroyed and I could never retrieve any of my work from my time abroad.

We use purely VR controls, and with specialised avatars we can get decently close to real sign, though without a specialised avatar you can still sign most signs.

Can you explain more about what you mean by "the recognition"? What were you trying to do?

So, the idea was a trainer.

It would show you signs and you had to try and imitate them.

The easiest, and best way, to store the signs, was by using the leap motion controller to record hand positions.

You then for the training bit it would play back that motion, and you had to copy it.

When copying it, your hand would appear on screen, and no matter how close we could get it - we even had a readout of all the joint vectors on screen, the vectors were how the hand "image" was recorded and stored to memory - we could never hit enough accuracy to get the program to say "you've done it, you've successfully signed the Letter A" or whatever.

Thinking back, we'd have to adjust the vectors to make them unit vectors (length 1, just direction) to normalise for different sized hands, and then to normalise the cartesian coordinates so that the vectors were in relation to a point on your hand rather than in relation to the controller itself being the (0,0,0) point.

This was rather difficult to do, and the last stumbling block, which we never got over.

After returning to the UK where we use BSL rather than ASL, I had intended to continue it, but as I said, my laptop got literally destroyed so I lost all the work.

I don't really think that's possible. IIRC some people a few year ago made gloves to translate ASL before understanding the language properly and it just didn't work well.

In all sign languages a big part of the language is the facial expression, the speed and where you sign it on the body which can't really be measured by the equipment you were using. I'm sure you could get it to work for very basic things but at that point videos alone should be fine.

It's a good idea but there are tons of hurdles to make it work if it's even possible at all.

Also, be careful making things like this for the Deaf. Those guys with the gloves really pissed off a bunch of Deaf people for some reason.

In all sign languages a big part of the language is the facial expression, the speed and where you sign it on the body which can't really be measured by the equipment you were using. I'm sure you could get it to work for very basic things but at that point videos alone should be fine.

The assignment topics were handed out, so we were told to make a Virtual/Augmented Reality ASL trainer and left to figure it out.

The leap motion was really the only commercially available product at the time which could do finger tracking rather than hand tracking, but it still had issues.

We realised pretty quickly that it wasn't suitable for what I'll call "conversational ASL" thus we only really focused on letter spelling which can solely be done with hand signs as far as I'm aware, but please correct me if I'm wrong.

So we went for more of a proof of concept than a finished product.

IIRC there are some letter signs which are essentially the exact same, just with your palm towards you vs away from you.

This caused issues, because as soon as fingers are occluded, it had no idea how to proceed.

We had a spec to follow for the class so we followed it, but it would be hella difficult to get by in life literally having to spell out everything you want to say, letter by letter.

To do something like we envisioned you'd need at least 2 controllers, wearing one of them so you can get both sides of your hand, plus something like a kinect which can track your overall body movements and facial expressions.

You're looking at unfathomable amounts of work but I believe in you

That work has been gone for the past 6 years.

The laptop with all the work on it was confiscated in a police investigation into a family member who had used my computers and at the end of the investigation it was deemed that it couldn't be returned so, among some other stuff including my old gaming pc, it was put into an industrial shredder.

They wouldn't even let me retrieve some important data under supervision.

That's fucked up. Hope you figure out how to replicate what you made before.

Is there a version of Esperanto for sign language? Do you think asl will come to dominate online communities?

Basically, is there a path way for you to communicate to BSL signers on-Line? Or will ask just be the language du jour?

There is an International Sign Language which I would say is closest to the idea of Esperanto but very few people use it because it removes all the culture from their respective languages and various other reasons I cannot remember right now.

A better idea that's kind of close to the idea of Esperanto is PSE, or Pigin Signed English. Regular ASL has the grammatical structure of French, making it a pain in the ass to learn for English speakers. PSE is ASL but in English grammatical structure with every word that doesn't convey necessary information removed. I think about 80% of all signers in America use PSE rather than ASL.

So a sentence in English converted to ASL would go from "My mom has brown hair and blue eyes." would become "My mom hair brown eyes blue." If you convert that to PSE it'd be "My mom have brown hair blue eyes". That's not a great example sentence because not a lot changes but it's all I can think of.

I doubt ASL will be dominating anything, VR is still not widespread and once it is I doubt ASL will be super popular.

VR really doesn't provide any way for ASL signers to communicate with BSL ones. They're as different as English and Spanish.

Why don’t you think ASL will be popular in VR? It’s my American bias but, aren’t there a shit ton of HoH Americans and America is a rich country who can afford VR tech stuff?

I mean, there are lots of HoH and Deaf people but its very hard to find people signing unless you know where to look. Also people don't generally learn second languages just out of the blue. There are exceptions to that rule obviously but I don't think many people will really care about learning it.

VrChat is incredible for the sign language community. One of my friends practice sign language in VrChat just like yourself. Are you familiar with LeapMotion? External hand sensor. I got to try it with a local study. Comparing it to the Index controllers. It was very impressive and I thought this would be a game changer for the sign language community. May I ask why you haven't tried or are not currently using the Index controller for signing?

I've never heard of LeapMotion

I'm not using index because the controllers alone cost more than 2 of my headsets and controllers combined so I'm not really going to drop all that money.

I have an avatar specifically for ASL that lets me do most of the important handshapes anyway.

I worked with the Leapmotion for a couple game jam-type events a few years ago. It worked insanely well at capturing hand movements.

Funnily enough, one of the teams I worked on took a Leapmotion tracker, and taped it onto the front of the Oculus Rift we had on-hand, and we had pretty much 1:1 hand tracking in the VR world we were making. Really impressive stuff.

That's super cool. I hope we get hand tracking like that in the future.

Know any other sign language, languages?

I know the alphabet for BSL/AUSLAN and in AUSLAN I can say "Yes/No", "Good/Bad", "Hungry/Sexy" and "Vegemite".

What's your ign?

Maybe I'll tell people later but I'm gonna stay anonymous for now

How long does it take to become semi fluent in sign language?

It depends on how fast you learn. I've seen people learn the alphabet in like 1 hour and I've seen it take people a few days. It took me about 2 months to be able to hold a decent conversation and now at the ~6 month mark I can convey practically anything I want to without too much trouble.

Thanks for the quick reply, I'm currently learning Japanese and it's taking forever especially since covid kept cancelling my fluency tests so I thought maybe I could try Sign Language since it's somewhat interesting

Sign is much easier than spoken languages IMO but I guess it depends on how good your muscle memory is.

Also, props for even attempting Japanese.

How long do you expect it to be (ballpark) until you can get a VR setup which, practically flawlessly, allows you to do the comprehensive, full range of ASL signs?

It really depends on what companies decide needs to be worked on when it comes to VR. Some are busy figuring out face/eye tracking and some are trying to figure out treadmill-type movement systems. They're not really focusing on hand tracking right now because there's not really a big need for it. It would be lovely but having perfectly tracked hands is such a tall order for something that won't really help with much gameplay-wise. I'd say at the very earliest maybe 4-5 years until they make something like that.

That being said, you can still use most signs in VR and those we can't we've edited so we can use them.

How are you enjoying it compared to an in-person class? Also how did you discover the VRChat community, were you signing before then or did you just see it and go cool!

I'm not a regular in the classes (I've gone to maybe one ever month) but they're much more enjoyable than IRL classes. My preferred learning style is by doing so I learned much more from just talking to as many people as I could.

As for how I discovered the community, I was sitting in a sink with a random guy and a taxidermied goose when the guy signed to me. I obviously didn't understand because I didn't know Sign yet so he brought me to a world where you can learn. I've used it almost every day since. (That was a sentence I never thought I'd say)

What do you think about the recent trend of deaf/hard of hearing plots that are now en vogue in Hollywood? My wife is a speech therapist at a deaf school so we've noticed this trend a little more than others. I do think it gives exposure to the community but it seems like everyone is jumping on the bandwagon lately and I'm not sure what kicked it off

I'm not sure what could have started it either, and being hearing myself I'm not sure my opinion really holds any weight. But... I don't really mind if they're jumping on the bandwagon. The more exposure for the Deaf community the better.

Are you able to help point me to any educational articles or videos which highlight how VR sign language works/is different from ASL?

This video is of the only ASL teacher in the community who isn't Deaf (She's hard of hearing) explaining the differences.

I just watched it and need to mention that she shows an old sign for "I", we've changed it.

How did you learn sign language?

It's actually easier than it sounds. I went to about 6 classes where they showed some signs but I learned almost everything from just conversing with people in sign. Whenever they use a sign you don't know you just ask what it is and expand your vocabulary, and go search for words you want to learn yourself.

Wow! I am a former ASL Interpreter and I remember anytime we had to watch someone on VHS (Yeah, I am old) or DVD (but not that old lol) and it would be so much harder than learning in person.

How does it work with Virtual Reality? any difference at all? Have you noticed an increase in skill level? is it easier? harder?

It has been a while since I have been out there interpreting but I would love to increase my skills again. Do you think for someone that has been out of the terp life it would be beneficial?

Eh, I'm Gen Z and had some VHS tapes growing up. I don't know if we were old-fashioned or poor, though. Lol

I'm actually Australian so I'd never have a chance to use ASL IRL but I can tell my friends/acquaintances experiences. They say VR sign pretty much just doesn't transfer back to IRL very well. They can get the gist of what is being said but rarely genuine comprehension.

Shaping the language into something that works in VR took a while because the medium of virtual reality is so limited, generic players only having access to having open hands, C hands, Fists/O hands, V hands, L hands, D hands, Thumbs up and Y hands. I have an avatar (Character) that gives me access to F hands, X hands, R hands, Middle finger down and Flat O's and I never really have problems with not being able to sign anything. We've changed signs like "Hair" from just grabbing some hair with your index and thumb to grabbing with the whole fist, we've changed "Like" to use a Y hand instead of a middle finger down, anything that's at least kinda close that isn't already a sign.

Fingerspelling is almost completely different excluding the letters C, D, G, L, O, V, Y and Z.

If you were to learn VR ASL it would practically be a different language, especially considering the buttons you need to press to change handshapes rather than having full control over your fingers but I'd recommend it. It's really enjoyable. If you wanted to get in contact with the community I can send you the Helping Hands discord. Everyone is happy to help newbies, not to mention if you mention you were an interpreter you'll get a ton of respect.

Do you have any encounters regarding bullying?

There are a bunch of people who will be Dicks about it, but that's to be expected. Honestly the thing that annoys me the most is that whenever anyone sees people signing who aren't learning themselves always go "HAHA GANG SIGNS HAHA" and they're confused when nobody laughs. I've heard that shit so many times. That and "HAHA NARUTO JUTSU".

Apologies if it's been asked already, but would there be more available signs with a controller different from the Quest 2, like the Valve Index with its finger tracking, and if so how much more complicated or how much easier would that be? I find the fact that VR ASL is a thing really fascinating, not something I even considered being available/prevalent in a virtual space. I should really get to learning to Sign one of these days. Thanks for the AMA!

Technically yes, index allows for more handshapes (B, F, K, U and K, mostly) but specialised avatars allow people with all controller types to have access to the important handshapes through combo gestures.

✋👏🤚✊🖐️🖖👋?

"YOUR MARRY YOUR S 5 [star trek sign language] HELLO"

That's exactly what I wanted to say!

Thanks for translating bro, pretty amusing :)

You're welcome! What an inspiring message it was. Honestly the MARRY part should have one hand rotated 90° but it's the closest sign I could think of

Now for my next trick, my impression of a snake:

✊✊✊✊✊✊✊✊✊✊✊✊✊✊✊✊✊✊✊

As an Australian I can say that this impression is pretty good

Are you learning Australian Sign Language?

I was planning on it, I learned the AUSLAN alphabet, hungry/sexy, yes/no, good/bad and Vegemite (the essentials) before deciding to focus on ASL.

CaN YOU SPEAK?

Ah you know the sacred chant

CaN YOU SPEAK?

LMAO JUST USE YOUR MIC