How do I change the name of an Azure Resource Group?

No, you cannot azure rename resource group, all you can do is move an existing resource group to a new resource group.

In the case of Azure Web Apps, you can only move all websites related resources in one invocation. This "all websites related resources" includes all websites, app hosting platforms, and certificates.

From the Portal

Use the "Move" tab for viewing resource groups

Clicking the "Move" tab will show this, which allow you to choose or create a new group:

Now Using Azure Powershell

use the Move-AzureRmResource powershell cmdlet.

The command for this:

Get-AzureRmResource -ResourceGroupName | Move-AzureRmResource -DestinationResourceGroupName

Using the Rest API:

Another way is to use the MoveResource Rest API or an ArmClient.

This is how you will make an API call:

POST

https:///subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/moveResources?api-version={api-version}

Where {resourceGroupName} is the source resource group. If you are using the ArmClient tool, then it will just take care of the endpoint.

Request Body:
{
   "targetResourceGroup": "/subscriptions/{subscriptionId}/resourceGroups/{targetResourceGroupNameName}",
   "resources":
   [  
     "/subscriptions/{id}/resourceGroups/{source}/providers/{namespace}/{type}/{name}",
     "/subscriptions/{id}/resourceGroups/{source}/providers/{namespace}/{type}/{name}"
   ]
}